Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...................XX.XXX.X.................................] duration:
2.637192848s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:07:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.637299418s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [....................XX.XXX.X................................] duration:
2.198497643s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:07:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.198521767s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.....................XX.XXX.X...............................] duration:
2.138686628s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:06:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.138704803s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [......................XX.XXX.X..............................] duration:
2.623495939s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:05:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.623516548s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.......................XX.XXX.X.............................] duration:
2.322294829s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:04:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.322317931s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [........................XX.XXX.X............................] duration:
2.599765288s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:04:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.599786889s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.........................XX.XXX.X...........................] duration:
2.618163691s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:03:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.618182474s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..........................XX.XXX.X..........................] duration:
2.721743558s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:02:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.72176594s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...........................XX.XXX.X.........................] duration:
2.37440362s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:01:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.374420191s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [............................XX.XXX.X........................] duration:
2.310415155s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2026-01-01 00:00:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.310441824s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.............................XX.XXX.X.......................] duration:
2.630063268s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:59:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.630085861s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..............................XX.XXX.X......................] duration:
2.572839371s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:58:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.572860941s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...............................XX.XXX.X.....................] duration:
2.693899658s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:58:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.69392759s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [................................XX.XXX.X....................] duration:
2.674743504s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:57:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.674766977s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.................................XX.XXX.X...................] duration:
2.181714575s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:56:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.181740684s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..................................XX.XXX.X..................] duration:
2.585553446s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:55:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.585573354s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...................................XX.XXX.X.................] duration:
2.59930724s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:54:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.599327859s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [....................................XX.XXX.X................] duration:
2.637567363s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:53:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.637589253s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.....................................XX.XXX.X...............] duration:
2.565148249s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:52:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.565173598s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [......................................XX.XXX.X..............] duration:
2.135870429s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:51:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.135895094s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.......................................XX.XXX.X.............] duration:
2.540517222s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:50:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.54053777s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [........................................XX.XXX.X............] duration:
2.604740131s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:50:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.604750941s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.........................................XX.XXX.X...........] duration:
2.671383424s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:49:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.671404073s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..........................................XX.XXX.X..........] duration:
2.649156612s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:49:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.649182581s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...........................................XX.XXX.X.........] duration:
2.603816857s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:48:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.603839619s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [............................................XX.XXX.X........] duration:
2.63536267s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:47:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.635387066s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.............................................XX.XXX.X.......] duration:
2.565739653s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:46:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.565771612s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..............................................XX.XXX.X......] duration:
2.138488584s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:45:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.138514032s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...............................................XX.XXX.X.....] duration:
2.277276124s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:45:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.277325195s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [................................................XX.XXX.X....] duration:
2.152691779s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:44:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.152726543s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [.................................................XX.XXX.X...] duration:
2.625744297s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:43:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.625756901s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [..................................................XX.XXX.X..] duration:
2.228917824s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:43:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.228936309s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [...................................................XX.XXX.X.] duration:
2.187731556s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:42:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 2.187743347s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [....................................................XX.XXX.X] duration:
1m0.125459172s trace:
https://jaeger.dev.storj.io/trace/452d3cc74373b393
FAIL (2025-12-31 23:42:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 16)
======LOGS======
...
Dec 31 23:41:49.829847249 (632h12m37.527415555s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Uploading piece...
Dec 31 23:41:50.945535964 (632h12m38.643104269s) [upload=556187]: Next returned nil segment
Dec 31 23:41:50.945545962 (632h12m38.643114278s) [upload=556187]: Waiting for error group managing segments...
Dec 31 23:41:52.092596174 (632h12m39.79016449s) [noise=true , address=89.149.201.147:4006 , piece_id=43EG4VB6BG3SB66XLGM3T7O63LMSZGMH2ITE4726JQSV7CXLCXEA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:34.528461577 (632h13m22.226029914s) [noise=true , address=81.171.14.70:4038 , piece_id=3EMLWZMDZ6SOUDALVBDYOKBUYARI7DKV3ZSPRV7VK5SSQHGMPPTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:35.876196480 (632h13m23.573764796s) [noise=true , address=81.171.14.70:4029 , piece_id=NUOGE57O7ROTLCJAZ74KQS23HKROOBYABWTASSCFYI635K63NOUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.585780771 (632h13m25.283349087s) [noise=true , address=81.171.14.70:4047 , piece_id=ZE57FIZ7CXXC7VLW6YZD576PXOOQHTTWQMKTGM6ONOWFWTMFP44Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:37.666430501 (632h13m25.363998838s) [noise=true , address=89.149.201.147:4021 , piece_id=OY3AIS5LP2CXKFLXP42UPEA547PRGP3GPYKYEBGUC37IEQ26KTYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.077738826 (632h13m25.775307172s) [noise=true , address=81.171.14.70:4044 , piece_id=P2RWSHV3R5M4FPBE4GYF4X2YW7LFTHVXQJDWTRDQJM66UPTIQOMQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:38.216423432 (632h13m25.913991768s) [noise=true , address=89.149.201.147:4009 , piece_id=BVBG7J4A6ZLOKIJY4ULARQZ6FV62J46D735ECYI33SQC6GY5WS2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:40.626259401 (632h13m28.323827746s) [noise=true , address=185.167.204.206:4045 , piece_id=WTESRDAFCMCK4AFPADSDW7SSVTPP45FK24D55EG7FFNRWKQIMVBA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.684393439 (632h13m29.381961745s) [noise=true , address=81.171.14.70:4039 , piece_id=LCH657GMV4CXGGOAUALI52TZIKZX2EWTFGVIJG75M4CMELD3YRDQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.784014482 (632h13m29.481582799s) [noise=true , address=185.167.204.206:4071 , piece_id=IN53N76O3OTOQSIP26RN6UUDCLZGP6J5JOEGIB6LP7264GSGJ4LQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:41.808190896 (632h13m29.505759202s) [noise=true , address=185.167.204.206:4037 , piece_id=3OI2IL37IDFTONKYZPPSZNHZDWZUVRTAQ4XKDZQVPXXZHCIICCJA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:43.142026890 (632h13m30.839595227s) [noise=true , address=89.149.201.147:4010 , piece_id=AHMVX7SBX5ZHGJ6T6KJFFO4NOAW6QN6ZZGR6CS4BBCQMKIAJOKSA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.672753895 (632h13m33.370322202s) [noise=true , address=185.167.204.206:4062 , piece_id=LFZAITYGNUCR6CCSALCR723T4FMUWN66PJ2XU2XOHEFDIW75KI6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.712124575 (632h13m33.40969288s) [noise=true , address=185.167.204.205:4024 , piece_id=GUQQDLKBRGVT5C6WWFMAKWPUXDY3SCELCVWOMQRK2A6FNRLGERDA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:45.763360618 (632h13m33.460928934s) [noise=true , address=81.171.14.70:4031 , piece_id=QPTX7FKRBCYHDTLQCCPZIBXJ7VRAOIMX5JOO6RQUZGJ7SIEBBFCQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:46.280135014 (632h13m33.97770332s) [noise=true , address=81.171.14.70:4026 , piece_id=6IY5YZVXMLOADAVGGI7IT6GNKDMAG5AJNVU2VVWPYVCIVMH7X2AA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: <nil>
Dec 31 23:42:49.189666602 (632h13m36.887234918s) [noise=true , address=185.167.204.205:4009 , piece_id=DAJ2W4JIFIZUJC65WOP52C7S42ZHDO7DIQA5QVBZVW7BKKALKYYA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: rpc: dial tcp 185.167.204.205:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189757221 (632h13m36.887325528s) [noise=true , address=185.167.204.206:4060 , piece_id=DOA7EYNAMTZP7GMPWSZTQMGGB2GF76TDDLIPHIASFBQ52W5I7J4A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: rpc: dial tcp 185.167.204.206:4060: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189897934 (632h13m36.88746628s) [noise=true , address=81.171.14.70:4042 , piece_id=TNXWRHLY6VIKZ5B3ZIOBQD2AL437YYHP5ZQRNFNDEQ2HS5MKKB2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: rpc: dial tcp 81.171.14.70:4042: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.189963506 (632h13m36.887531822s) [noise=true , address=185.167.204.205:4025 , piece_id=EFX5UJY6CTOCMON5BKPQM3CT3L4CBA6RFEO2PLL4NTH5YXNATUYQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1cnhWXZCgZz567E5PAaPyfNKkmsSa4imaWPmfW2ir7U48gYEkp, address:185.167.204.205:4025): protocol: rpc: dial tcp 185.167.204.205:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190181704 (632h13m36.88775001s) [noise=true , address=185.167.204.206:4063 , piece_id=ZNP7X7MOOT2FOHMOEFCOY4CP5RUTBO3QWW3OJO2JW34OOORO2P6A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: rpc: dial tcp 185.167.204.206:4063: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190244741 (632h13m36.887813057s) [noise=true , address=89.149.201.147:4013 , piece_id=JLEIBUCOXKBDNCX4BLHJGAEMJXHAMPBI2CBJL4URJTX5SCGX7QTA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: rpc: dial tcp 89.149.201.147:4013: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190457339 (632h13m36.888025645s) [noise=true , address=185.167.204.206:4048 , piece_id=APIMMJXUKXAQOWACAOTLM6HBV3VQ4CFI3E37QE75ICMJ74BLE6EA, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1sY4wHBkSVUDWWg8T8Vm1hpRbbpb3ttqnqDVY8uWyG27gqBwT9, address:185.167.204.206:4048): protocol: write tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; close tcp 5.161.89.252:29008->185.167.204.206:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190654597 (632h13m36.888222893s) [noise=true , address=185.167.204.205:4008 , piece_id=KC4LOM6TGVY7FVD7O5WZZUCMFJE3PXGKPKXEQ4HV7DJNKJTUUG2A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: rpc: dial tcp 185.167.204.205:4008: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.190879228 (632h13m36.888447533s) [noise=true , address=185.167.204.205:4004 , piece_id=NWOFRBSVSSPVDHY5YMFGVLN2O3TK37DTSC2STLE235G6PODPBJUQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: read tcp 5.161.89.252:21956->185.167.204.205:4004: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191034467 (632h13m36.888602773s) [noise=true , address=185.167.204.205:4030 , piece_id=3JES3KIDGX4WQDZUNR3GW2JZQUJ3N5PPFYR4XQK7KJKN3V5ZLAKQ, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:11082->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191273213 (632h13m36.888841519s) [noise=true , address=185.167.204.206:4043 , piece_id=RO3U776CTZVCC3ZINF3VHDSKDWJWK53EKBIZT4LWHIJSY4SRJK3A, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1EtbRMgkYPVZWSW6hTfGghxDoetX26Fm1Rw97n49jR1oMYt1kb, address:185.167.204.206:4043): protocol: read tcp 5.161.89.252:50484->185.167.204.206:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191308529 (632h13m36.888876835s) [noise=true , address=185.167.204.206:4040 , piece_id=PV3JC3E36RLMVTREPSO6PAAXNWHD43DNARLDDWD236LGOVKHBH7Q, seg_pos={0 0}, upload=556187]: Done uploading piece. err: ecclient: upload failed (node:1F6P8VfgAdXdcXykRHWpik5CDN4aWj7LKrugb8uAThZpAchkdT, address:185.167.204.206:4040): protocol: read tcp 5.161.89.252:60810->185.167.204.206:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:42:49.191329418 (632h13m36.888897724s) [seg_pos={0 0}, upload=556187]: Done waiting for segment. successful: 16 optimal: 26 errs: <nil>
Dec 31 23:42:49.313678458 (632h13m37.011246774s) [upload=556187]: Done uploading segments...
Dec 31 23:42:49.313702132 (632h13m37.011270448s) [upload=556187]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 16)
Duration: 1m0.125476304s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [.....................................................XX.XXX.] duration:
2.205197525s trace:
https://jaeger.dev.storj.io/trace/4444ccc23f47ad8a
FAIL (2025-12-31 23:41:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 19)
======LOGS======
3:39:59.634030981 (632h10m47.331599317s) [noise=true , address=185.167.204.205:4005 , piece_id=MMRHYET33BXS72NUGWOJB6DKXAMOQZ7F5H7J26LJIQXGR2MQEKGA, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.634570309 (632h10m47.332138644s) [noise=true , address=185.167.204.205:4012 , piece_id=LEEOJXHWPLK3MK2XE4W7DPB4N4GFFP6QXRUBZ7XGZCZCQ3T6XBCQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.635142367 (632h10m47.332710703s) [noise=true , address=81.171.14.70:4029 , piece_id=IS3ZPY6MXHN4NPW365WDSYFVLKBTXLO3TADZU3AOCCV7LHEUGLNQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.635649245 (632h10m47.33321758s) [noise=true , address=185.167.204.206:4062 , piece_id=UZE7UEA5ZD2753N3HEFELDDQSKY2WBSMM6QLOVPGXG26ERGRLIEQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.635879935 (632h10m47.333448271s) [noise=true , address=81.171.14.70:4037 , piece_id=LEC6VOEGANM2GSOEWDIOPX7EV4HJ5XJZ43OQ2FKWTHNYA5IX7EXQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:40:00.422876051 (632h10m48.120444386s) [upload=556160]: Next returned nil segment
Dec 31 23:40:00.422887913 (632h10m48.120456249s) [upload=556160]: Waiting for error group managing segments...
Dec 31 23:40:01.137245141 (632h10m48.834813458s) [noise=true , address=81.171.14.70:4034 , piece_id=4XZWBMOUO67T7WEG6VX75SRYYKJN2KJME5HR3QT33OQXG2HFFYTA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:01.256821602 (632h10m48.954389908s) [noise=true , address=185.167.204.206:4048 , piece_id=W36UJBQPQ6HRX7CQO3QUIBMP24OI2LYHFG434IWQ3XPGAYRQVWFQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:01.996758787 (632h10m49.694327122s) [noise=true , address=81.171.14.70:4035 , piece_id=TRHQL3EBEB56EVEWWSF6SL4EKHM5F34DHC5IFO4TNEKGV7243OSQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:02.529784991 (632h10m50.227353307s) [noise=true , address=81.171.14.70:4047 , piece_id=X2XKNNDDWRDSRMUR43W5GMUH5YKSOUSMUX6YKIQSIEQUVQBSSLAA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:02.632016540 (632h10m50.329584846s) [noise=true , address=81.171.14.70:4037 , piece_id=LEC6VOEGANM2GSOEWDIOPX7EV4HJ5XJZ43OQ2FKWTHNYA5IX7EXQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:49.626235745 (632h11m37.323804051s) [noise=true , address=89.149.201.147:4016 , piece_id=BDN2EZ6BNXFFD6APNSKCAGI3BXTCGDESMMVNIVMGSVELNNAJ3ZVQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:51.662470869 (632h11m39.360039204s) [noise=true , address=81.171.14.70:4029 , piece_id=IS3ZPY6MXHN4NPW365WDSYFVLKBTXLO3TADZU3AOCCV7LHEUGLNQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:53.183454622 (632h11m40.881022958s) [noise=true , address=185.167.204.206:4056 , piece_id=O2GVDKLWAK4D66AXSVRTPINBGRQFQVHO3SH2D7J4GAA7BT2EARWQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:53.810975978 (632h11m41.508544314s) [noise=true , address=89.149.201.147:4006 , piece_id=KFSNLM4D5LO5BRTYICNDCIPWZ62P7QKB5TNFRX64PSU5TOJTF3WA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12YTf9jvAKd4gVTxTZC6gGLdXC17ygbrqvqoKtmPbtHyc65YjDU, address:89.149.201.147:4006): protocol: read tcp 5.161.89.252:53416->89.149.201.147:4006: read: connection reset by peer; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:55.143399799 (632h11m42.840968136s) [noise=true , address=185.167.204.206:4066 , piece_id=EDRLVEXNEVT2ZXGX5E3HVAJP7K65KGEW7N5MP3XQJYBKMWKXGRAQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.153494307 (632h11m42.851062643s) [noise=true , address=185.167.204.206:4063 , piece_id=Q7OKLTOLPWOUCFDA2LNFWQ3SO4CELIXWQHB343Y3IVYNAEJB77VQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.227998859 (632h11m42.925567194s) [noise=true , address=81.171.14.70:4025 , piece_id=IDDRT2OW6T4R5WCSV3EZ4MI6ROQX7GCP3QXUXFX5ULVULQIRPYOQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.297548745 (632h11m42.99511708s) [noise=true , address=81.171.14.70:4041 , piece_id=N2EB2X234F77BSPBV3XCXJJ3PJ76ZBY53KE5VOGQBB3VBRBMH5PQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.301713094 (632h11m42.999281431s) [noise=true , address=89.149.201.147:4013 , piece_id=PJKPKS5DBSOF353ZKGGBZWHGU7AQ4QF7E5DHMVVFIZQ33V27U73A, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.305236275 (632h11m43.002804612s) [noise=true , address=89.149.201.147:4018 , piece_id=L5VOSJPULIICZAIZLW3W6MUAVPKKRHL6RYG56UNXUQ4PH3VLORYQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:56.930090260 (632h11m44.627658607s) [noise=true , address=185.167.204.206:4054 , piece_id=WIRF7GDTDIKOG4N54VLVAMR3PZBVWIGG36KMTEMXFFW7PJLQD2EA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.136504803 (632h11m44.834073139s) [noise=true , address=185.167.204.205:4010 , piece_id=UK6N2SPFPLUXHA4BJJ4U2OQ6Q4LOQRCJWRWYOANHJLHMXF35MFAA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.188266589 (632h11m44.885834935s) [noise=true , address=185.167.204.206:4053 , piece_id=FL43JN76JS3LEVAJ6C6W4BIYNORZCGRGDK3BVV7GSXVEYDSYZXBA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.204019206 (632h11m44.901587552s) [noise=true , address=185.167.204.206:4043 , piece_id=NFDT6SILOWUDTIFJEMHMXQECMSIQ7PC2VRTT63H7IBKQBCOGWKKA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.379251316 (632h11m45.076819652s) [noise=true , address=185.167.204.206:4071 , piece_id=XMKC634RCZTYSS5R7KPFZ3MH7AZHTH6RGZBNLEVORMSZCC4HESSA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:59.221881428 (632h11m46.919449775s) [noise=true , address=89.149.201.147:4010 , piece_id=BEGHMN364SDP4JIYRMDI6HAQ5QYUS3IRNLYJY2CACUTD2IBIUNTA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: expected piece hash; context deadline exceeded; EOF
Dec 31 23:40:59.221915252 (632h11m46.919483588s) [noise=true , address=81.171.14.70:4032 , piece_id=RUWYHUFZHKNNQ5FDJPWQJ4JG652DOXWWR5Z7BJ6KC4NCKH6E2WXQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: rpc: dial tcp 81.171.14.70:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222230381 (632h11m46.919798727s) [noise=true , address=81.171.14.70:4030 , piece_id=NPSSFKK7FHH3HMMWMRL2CGDGR5X5DEGDAJKI5CDDSXJDIUYOD54Q, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 5.161.89.252:61740->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222307274 (632h11m46.91987561s) [noise=true , address=81.171.14.70:4048 , piece_id=FETLL6RYCSE2FHNQRWRLUM26NNUJT2BU2LG7W4FEAP4LOHH5TT2A, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222489845 (632h11m46.920058182s) [noise=true , address=89.149.201.147:4020 , piece_id=JSSLDKCCJUJOBLIF7QNPQKH7FAYGDYYF2HX2LRV3FAKFUOAMCBAQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: read tcp 5.161.89.252:20756->89.149.201.147:4020: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222592787 (632h11m46.920161123s) [noise=true , address=185.167.204.206:4062 , piece_id=UZE7UEA5ZD2753N3HEFELDDQSKY2WBSMM6QLOVPGXG26ERGRLIEQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: read tcp 5.161.89.252:27294->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222750903 (632h11m46.920319239s) [noise=true , address=185.167.204.205:4005 , piece_id=MMRHYET33BXS72NUGWOJB6DKXAMOQZ7F5H7J26LJIQXGR2MQEKGA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, address:185.167.204.205:4005): protocol: read tcp 5.161.89.252:19736->185.167.204.205:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222853165 (632h11m46.920421501s) [noise=true , address=185.167.204.205:4012 , piece_id=LEEOJXHWPLK3MK2XE4W7DPB4N4GFFP6QXRUBZ7XGZCZCQ3T6XBCQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:48738->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222892387 (632h11m46.920460723s) [seg_pos={0 0}, upload=556160]: Done waiting for segment. successful: 19 optimal: 26 errs: <nil>
Dec 31 23:40:59.330785754 (632h11m47.0283541s) [upload=556160]: Done uploading segments...
Dec 31 23:40:59.330818195 (632h11m47.02838653s) [upload=556160]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 19)
Duration: 2.205211491s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [......................................................XX.XXX] duration:
1m0.111684411s trace:
https://jaeger.dev.storj.io/trace/4444ccc23f47ad8a
FAIL (2025-12-31 23:40:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 19)
======LOGS======
3:39:59.634030981 (632h10m47.331599317s) [noise=true , address=185.167.204.205:4005 , piece_id=MMRHYET33BXS72NUGWOJB6DKXAMOQZ7F5H7J26LJIQXGR2MQEKGA, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.634570309 (632h10m47.332138644s) [noise=true , address=185.167.204.205:4012 , piece_id=LEEOJXHWPLK3MK2XE4W7DPB4N4GFFP6QXRUBZ7XGZCZCQ3T6XBCQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.635142367 (632h10m47.332710703s) [noise=true , address=81.171.14.70:4029 , piece_id=IS3ZPY6MXHN4NPW365WDSYFVLKBTXLO3TADZU3AOCCV7LHEUGLNQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.635649245 (632h10m47.33321758s) [noise=true , address=185.167.204.206:4062 , piece_id=UZE7UEA5ZD2753N3HEFELDDQSKY2WBSMM6QLOVPGXG26ERGRLIEQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:39:59.635879935 (632h10m47.333448271s) [noise=true , address=81.171.14.70:4037 , piece_id=LEC6VOEGANM2GSOEWDIOPX7EV4HJ5XJZ43OQ2FKWTHNYA5IX7EXQ, seg_pos={0 0}, upload=556160]: Uploading piece...
Dec 31 23:40:00.422876051 (632h10m48.120444386s) [upload=556160]: Next returned nil segment
Dec 31 23:40:00.422887913 (632h10m48.120456249s) [upload=556160]: Waiting for error group managing segments...
Dec 31 23:40:01.137245141 (632h10m48.834813458s) [noise=true , address=81.171.14.70:4034 , piece_id=4XZWBMOUO67T7WEG6VX75SRYYKJN2KJME5HR3QT33OQXG2HFFYTA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:01.256821602 (632h10m48.954389908s) [noise=true , address=185.167.204.206:4048 , piece_id=W36UJBQPQ6HRX7CQO3QUIBMP24OI2LYHFG434IWQ3XPGAYRQVWFQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:01.996758787 (632h10m49.694327122s) [noise=true , address=81.171.14.70:4035 , piece_id=TRHQL3EBEB56EVEWWSF6SL4EKHM5F34DHC5IFO4TNEKGV7243OSQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:02.529784991 (632h10m50.227353307s) [noise=true , address=81.171.14.70:4047 , piece_id=X2XKNNDDWRDSRMUR43W5GMUH5YKSOUSMUX6YKIQSIEQUVQBSSLAA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:02.632016540 (632h10m50.329584846s) [noise=true , address=81.171.14.70:4037 , piece_id=LEC6VOEGANM2GSOEWDIOPX7EV4HJ5XJZ43OQ2FKWTHNYA5IX7EXQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:49.626235745 (632h11m37.323804051s) [noise=true , address=89.149.201.147:4016 , piece_id=BDN2EZ6BNXFFD6APNSKCAGI3BXTCGDESMMVNIVMGSVELNNAJ3ZVQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:51.662470869 (632h11m39.360039204s) [noise=true , address=81.171.14.70:4029 , piece_id=IS3ZPY6MXHN4NPW365WDSYFVLKBTXLO3TADZU3AOCCV7LHEUGLNQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:53.183454622 (632h11m40.881022958s) [noise=true , address=185.167.204.206:4056 , piece_id=O2GVDKLWAK4D66AXSVRTPINBGRQFQVHO3SH2D7J4GAA7BT2EARWQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:53.810975978 (632h11m41.508544314s) [noise=true , address=89.149.201.147:4006 , piece_id=KFSNLM4D5LO5BRTYICNDCIPWZ62P7QKB5TNFRX64PSU5TOJTF3WA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12YTf9jvAKd4gVTxTZC6gGLdXC17ygbrqvqoKtmPbtHyc65YjDU, address:89.149.201.147:4006): protocol: read tcp 5.161.89.252:53416->89.149.201.147:4006: read: connection reset by peer; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:55.143399799 (632h11m42.840968136s) [noise=true , address=185.167.204.206:4066 , piece_id=EDRLVEXNEVT2ZXGX5E3HVAJP7K65KGEW7N5MP3XQJYBKMWKXGRAQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.153494307 (632h11m42.851062643s) [noise=true , address=185.167.204.206:4063 , piece_id=Q7OKLTOLPWOUCFDA2LNFWQ3SO4CELIXWQHB343Y3IVYNAEJB77VQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.227998859 (632h11m42.925567194s) [noise=true , address=81.171.14.70:4025 , piece_id=IDDRT2OW6T4R5WCSV3EZ4MI6ROQX7GCP3QXUXFX5ULVULQIRPYOQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.297548745 (632h11m42.99511708s) [noise=true , address=81.171.14.70:4041 , piece_id=N2EB2X234F77BSPBV3XCXJJ3PJ76ZBY53KE5VOGQBB3VBRBMH5PQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.301713094 (632h11m42.999281431s) [noise=true , address=89.149.201.147:4013 , piece_id=PJKPKS5DBSOF353ZKGGBZWHGU7AQ4QF7E5DHMVVFIZQ33V27U73A, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:55.305236275 (632h11m43.002804612s) [noise=true , address=89.149.201.147:4018 , piece_id=L5VOSJPULIICZAIZLW3W6MUAVPKKRHL6RYG56UNXUQ4PH3VLORYQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:56.930090260 (632h11m44.627658607s) [noise=true , address=185.167.204.206:4054 , piece_id=WIRF7GDTDIKOG4N54VLVAMR3PZBVWIGG36KMTEMXFFW7PJLQD2EA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.136504803 (632h11m44.834073139s) [noise=true , address=185.167.204.205:4010 , piece_id=UK6N2SPFPLUXHA4BJJ4U2OQ6Q4LOQRCJWRWYOANHJLHMXF35MFAA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.188266589 (632h11m44.885834935s) [noise=true , address=185.167.204.206:4053 , piece_id=FL43JN76JS3LEVAJ6C6W4BIYNORZCGRGDK3BVV7GSXVEYDSYZXBA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.204019206 (632h11m44.901587552s) [noise=true , address=185.167.204.206:4043 , piece_id=NFDT6SILOWUDTIFJEMHMXQECMSIQ7PC2VRTT63H7IBKQBCOGWKKA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:57.379251316 (632h11m45.076819652s) [noise=true , address=185.167.204.206:4071 , piece_id=XMKC634RCZTYSS5R7KPFZ3MH7AZHTH6RGZBNLEVORMSZCC4HESSA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: <nil>
Dec 31 23:40:59.221881428 (632h11m46.919449775s) [noise=true , address=89.149.201.147:4010 , piece_id=BEGHMN364SDP4JIYRMDI6HAQ5QYUS3IRNLYJY2CACUTD2IBIUNTA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: expected piece hash; context deadline exceeded; EOF
Dec 31 23:40:59.221915252 (632h11m46.919483588s) [noise=true , address=81.171.14.70:4032 , piece_id=RUWYHUFZHKNNQ5FDJPWQJ4JG652DOXWWR5Z7BJ6KC4NCKH6E2WXQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: rpc: dial tcp 81.171.14.70:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222230381 (632h11m46.919798727s) [noise=true , address=81.171.14.70:4030 , piece_id=NPSSFKK7FHH3HMMWMRL2CGDGR5X5DEGDAJKI5CDDSXJDIUYOD54Q, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 5.161.89.252:61740->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222307274 (632h11m46.91987561s) [noise=true , address=81.171.14.70:4048 , piece_id=FETLL6RYCSE2FHNQRWRLUM26NNUJT2BU2LG7W4FEAP4LOHH5TT2A, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: rpc: dial tcp 81.171.14.70:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222489845 (632h11m46.920058182s) [noise=true , address=89.149.201.147:4020 , piece_id=JSSLDKCCJUJOBLIF7QNPQKH7FAYGDYYF2HX2LRV3FAKFUOAMCBAQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12YeKHJH7CoouLFJYENJhzVLNkA32hJaqjLz3dXhr78eaSGtcic, address:89.149.201.147:4020): protocol: read tcp 5.161.89.252:20756->89.149.201.147:4020: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222592787 (632h11m46.920161123s) [noise=true , address=185.167.204.206:4062 , piece_id=UZE7UEA5ZD2753N3HEFELDDQSKY2WBSMM6QLOVPGXG26ERGRLIEQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12BL6XLqsoUGm2FxrTgYTxLZakt2u2CAhZJG82oJ31eaE4vTtMr, address:185.167.204.206:4062): protocol: read tcp 5.161.89.252:27294->185.167.204.206:4062: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222750903 (632h11m46.920319239s) [noise=true , address=185.167.204.205:4005 , piece_id=MMRHYET33BXS72NUGWOJB6DKXAMOQZ7F5H7J26LJIQXGR2MQEKGA, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, address:185.167.204.205:4005): protocol: read tcp 5.161.89.252:19736->185.167.204.205:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222853165 (632h11m46.920421501s) [noise=true , address=185.167.204.205:4012 , piece_id=LEEOJXHWPLK3MK2XE4W7DPB4N4GFFP6QXRUBZ7XGZCZCQ3T6XBCQ, seg_pos={0 0}, upload=556160]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:48738->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:40:59.222892387 (632h11m46.920460723s) [seg_pos={0 0}, upload=556160]: Done waiting for segment. successful: 19 optimal: 26 errs: <nil>
Dec 31 23:40:59.330785754 (632h11m47.0283541s) [upload=556160]: Done uploading segments...
Dec 31 23:40:59.330818195 (632h11m47.02838653s) [upload=556160]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 19)
Duration: 1m0.111703326s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
4 out of 60 is failed (threshold=3) [.......................................................XX.XX] duration:
1m0.106830629s trace:
https://jaeger.dev.storj.io/trace/4b97442795e56579
FAIL (2025-12-31 23:39:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 6)
======LOGS======
r: ecclient: upload failed (node:1pAUTtyVrPw8FCi8oTE1dsU8avWdehnDdCdtYN4P3zedYoBfiX, address:185.167.204.205:4015): protocol: write tcp 5.161.89.252:39916->185.167.204.205:4015: use of closed network connection; write tcp 5.161.89.252:39916->185.167.204.205:4015: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39916->185.167.204.205:4015: use of closed network connection
Dec 31 23:39:59.070676390 (632h10m46.768244697s) [noise=true , address=89.149.201.147:4016 , piece_id=DNOKUMJR5WGQEL5MHXZFFMJUTOZC7RCFSE7O46XAEUH4XRG42OTA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5, address:89.149.201.147:4016): protocol: read tcp 5.161.89.252:51920->89.149.201.147:4016: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.070694855 (632h10m46.768263162s) [noise=true , address=185.167.204.206:4058 , piece_id=ZHWOW7UU7AAZIGJ4KH22JC5DVHDYAXUMU6NLUNHGY6XPQZZKIG7A, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 5.161.89.252:24794->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.070884749 (632h10m46.768453065s) [noise=true , address=185.167.204.206:4055 , piece_id=5HNXJTXAKKAWZG4OWCO4MLFNXLF6BY64EMW6RDKQKXHYTLFHHN7A, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: read tcp 5.161.89.252:32460->185.167.204.206:4055: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.070976962 (632h10m46.768545278s) [noise=true , address=81.171.14.70:4027 , piece_id=Z5IV32CFTXGRRLHL4UUYEBRGBQTRAU25PH7I77KGUAUPKQC6VTIQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:31220->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071058694 (632h10m46.768627001s) [noise=true , address=81.171.14.70:4032 , piece_id=5XKBCZPYLKBGINIOK6DMXHIZT64WBTE3F7AOKYRZ4AO3TO5IWETQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: read tcp 5.161.89.252:35162->81.171.14.70:4032: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071068543 (632h10m46.768636849s) [noise=true , address=81.171.14.70:4046 , piece_id=A6SBU6NECH77SL57TZHZIFMNQM64NFTRAMML7E7O3UWYKS47PAFA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 5.161.89.252:17968->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071183087 (632h10m46.768751393s) [noise=true , address=89.149.201.147:4005 , piece_id=KOHZY3PTUYJVNOZEFYOTUFUCCXDAZXFN43EQFBILE6HVEZK7BWXA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 5.161.89.252:31816->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071213523 (632h10m46.76878183s) [noise=true , address=81.171.14.70:4033 , piece_id=AUUQLK4P5HRAN4XB77APYBOXCQ32KBCSG7KBIQVPM2ZACZBEFIPA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 5.161.89.252:45658->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071263087 (632h10m46.768831393s) [noise=true , address=185.167.204.206:4045 , piece_id=25Y5NPLCPF4VUCJ7ON6B4ETCLOH7SLX6QPR5EE5BA4IHZPA4TKYA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq, address:185.167.204.206:4045): protocol: read tcp 5.161.89.252:37202->185.167.204.206:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071306948 (632h10m46.768875264s) [noise=true , address=185.167.204.206:4060 , piece_id=L5DZFRYREZRO3GPI67DQWZPLCE2AWUUC7GEXDAKA4M3T427RZTPQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 5.161.89.252:55032->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071342003 (632h10m46.76891031s) [noise=true , address=89.149.201.147:4003 , piece_id=YZHTNK7EUDDJ2YUQPYJ46AHL6F5UEK3Q3ICW54DKHGLSUMYWRJHQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:134hab8xQdNGgLGN1nL6HZhXTX2Ysfi3zUdxbxEYmFjyZopqhX, address:89.149.201.147:4003): protocol: read tcp 5.161.89.252:60946->89.149.201.147:4003: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071382760 (632h10m46.768951076s) [noise=true , address=185.167.204.206:4052 , piece_id=CYHLZU54YYSCK3FO4A24AUMTHDBIEZHPN2ZLUDYTZTZWOVG2USMA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12Xr3UmaAcshmxZMecu5i7VHA6LJxiZm9u6wngapEma3hz3tNmn, address:185.167.204.206:4052): protocol: read tcp 5.161.89.252:23918->185.167.204.206:4052: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071417936 (632h10m46.768986252s) [noise=true , address=185.167.204.205:4028 , piece_id=NXM2553CDQXBR2L4WCNGO5323ELT7EMA4U6DMSLRI2MM6NWZYOJQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: read tcp 5.161.89.252:14006->185.167.204.205:4028: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071460384 (632h10m46.7690287s) [noise=true , address=185.167.204.206:4063 , piece_id=W2SB7BYMDJ6GNILAJE6YO5TKS5ZRYGFDGJSQHDRMVBN2GUMO6TEA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: read tcp 5.161.89.252:43298->185.167.204.206:4063: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071494559 (632h10m46.769062875s) [noise=true , address=185.167.204.205:4026 , piece_id=FAU2RP4PDEG4SGIJHXACYMTFQ5L6WVIXAE4PYNHCVMRJQ4J4ER5Q, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: read tcp 5.161.89.252:34422->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071537830 (632h10m46.769106136s) [noise=true , address=185.167.204.205:4007 , piece_id=U3XMRPUHDSTRDSYZVYDW4DWYY3BH6JL36RJ4MMMFPBUFUPAKKJAQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: read tcp 5.161.89.252:29682->185.167.204.205:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071570882 (632h10m46.769139188s) [noise=true , address=185.167.204.206:4046 , piece_id=NOD2BD7IG4XR3ZFX2ACU6XRR7MGGV5TZBKTHMDYS3S4RAHMOZXDA, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: read tcp 5.161.89.252:17092->185.167.204.206:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071606197 (632h10m46.769174513s) [noise=true , address=81.171.14.70:4048 , piece_id=TQB753EHHNOOH3T7DJMGTZVGN3V3JGUJ4AM4G2RVHQ42WDBPTT7Q, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: read tcp 5.161.89.252:36644->81.171.14.70:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071612378 (632h10m46.769180695s) [noise=true , address=89.149.201.147:4017 , piece_id=XK3IZZAGUDFTZHHZT77GQKQDU4GIGNMZWRFTNM5AVTFIMY2YYFUQ, seg_pos={0 0}, upload=556144]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: read tcp 5.161.89.252:34128->89.149.201.147:4017: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:39:59.071631204 (632h10m46.76919952s) [seg_pos={0 0}, upload=556144]: Done waiting for segment. successful: 6 optimal: 26 errs: <nil>
Dec 31 23:39:59.174420535 (632h10m46.871988911s) [upload=556144]: Done uploading segments...
Dec 31 23:39:59.174454828 (632h10m46.872023194s) [upload=556144]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 6)
Duration: 1m0.106843043s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [........................................................XX.X] duration:
1m0.10594458s trace:
https://jaeger.dev.storj.io/trace/2eb74e24d249f356
FAIL (2025-12-31 23:38:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 8)
======LOGS======
9 (632h9m8.047057426s) [noise=true , address=185.167.204.205:4020 , piece_id=OWUT6RLDOQPNUFSTSK2IKPH3EMCKPNMNGZAVVTAL25N4XCBTZNCA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: <nil>
Dec 31 23:38:40.791000734 (632h9m28.48856905s) [noise=true , address=81.171.14.70:4045 , piece_id=BWK6RHN3A6KSHRKVYK6LY3VUW2JQU2T5D22MHGOCESWAMXFZBK3Q, seg_pos={0 0}, upload=556129]: Done uploading piece. err: <nil>
Dec 31 23:38:58.912375901 (632h9m46.609944237s) [noise=true , address=81.171.14.70:4029 , piece_id=DG6ZR2ZLWG3RX7UFXXEG7NLM36NXNX7VZCFG35QHL2P4REWNWDUA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 31 23:38:58.912618403 (632h9m46.610186719s) [noise=true , address=81.171.14.70:4026 , piece_id=MDEOBCNWX2ELAI6CW33BKLBHLTS3RW33OAN4PSVP7XJKZ7GYYKXQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: read tcp 5.161.89.252:54208->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.912634513 (632h9m46.61020285s) [noise=true , address=89.149.201.147:4019 , piece_id=SWCEKRO4SUAODID4LO6ZHB5LFIR5RZPIBY25V62TIGHZKQVTIGTA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:12wgZQR7Ezekmvf9cNwCZ8CmZBjZgoqWC3N2UE3n4YQSmBsqQQ2, address:89.149.201.147:4019): protocol: rpc: dial tcp 89.149.201.147:4019: i/o timeout; context deadline exceeded
Dec 31 23:38:58.912729971 (632h9m46.610298287s) [noise=true , address=185.167.204.205:4036 , piece_id=HU7Z2JVPOGXZOSGJH7I3VVFFQACM3JSDFQA3BXRBUP4K2DTUJ3LA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:12Mngc2aRWoo1ZQGktFfA4DauhKyEFcbRk6dg7nYa1jZUg2srMV, address:185.167.204.205:4036): protocol: read tcp 5.161.89.252:9478->185.167.204.205:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.912814079 (632h9m46.610382395s) [noise=true , address=81.171.14.70:4039 , piece_id=MJEKWQHE5TGGOJZVUBJSTJ6YSEHPNVDN3LWE34IOFUCXDRKWL6QA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: read tcp 5.161.89.252:47724->81.171.14.70:4039: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.912830891 (632h9m46.610399277s) [noise=true , address=81.171.14.70:4033 , piece_id=F5DIV2XUGNGEBCV7JZNUVG6QEU2S3P4QQ4U4JW5ONPOAFXFFG26Q, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: expected piece hash; context deadline exceeded; EOF
Dec 31 23:38:58.912898436 (632h9m46.610466743s) [noise=true , address=81.171.14.70:4032 , piece_id=CXC6HOTT3XCWKLKELPBMD5EL4CQJ4FCYWVWFM5Y5OLKS5XV7XS7Q, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: read tcp 5.161.89.252:33858->81.171.14.70:4032: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.912980159 (632h9m46.610548476s) [noise=true , address=185.167.204.206:4038 , piece_id=JN2DKMW4J5UETIZMMX4FEWR5HODD3ANG74O4SCNUJBBNYSU752LQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: read tcp 5.161.89.252:21624->185.167.204.206:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913060009 (632h9m46.610628325s) [noise=true , address=185.167.204.206:4039 , piece_id=53MJDIAW5GISEQR4VIBNVRVXI3RIMVKTS5QOWXWIH53XJUSPDLZQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:12FV1y64fPXx14kCr3Tng2sYqXdsxaPeRJXmNVJLCVKQNkCo3Y9, address:185.167.204.206:4039): protocol: read tcp 5.161.89.252:26368->185.167.204.206:4039: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913138957 (632h9m46.610707303s) [noise=true , address=185.167.204.205:4009 , piece_id=DQ3BCSD4ATV32IC3NM2NBGRJOOH2S5JYJLLXK5EE2RM4KRHDZQZA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: expected piece hash; context deadline exceeded; EOF
Dec 31 23:38:58.913168611 (632h9m46.610736927s) [noise=true , address=185.167.204.206:4068 , piece_id=3LNPJ2W7YCR57B5VZCHAOIGPQ72O7LTVRO27G4REYSORCT4YDHIQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1sqDsiZKL7g4FbDyc9Ew2ojD1tTLzj4B19ma8hrCvWoTCy4j4D, address:185.167.204.206:4068): protocol: read tcp 5.161.89.252:38676->185.167.204.206:4068: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913301639 (632h9m46.610869946s) [noise=true , address=89.149.201.147:4013 , piece_id=53LOPCVM3572N7OT7DSHSWGW46Y73SCODNPTAV57ME25L3EDEJEQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: read tcp 5.161.89.252:48950->89.149.201.147:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913419511 (632h9m46.610987847s) [noise=true , address=185.167.204.206:4063 , piece_id=GV22LSJIHTWQJRS7DK6MUMM52LYPJI7XM42CPWW4OBPRPAVZBRWA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: read tcp 5.161.89.252:41972->185.167.204.206:4063: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913520288 (632h9m46.611088604s) [noise=true , address=89.149.201.147:4010 , piece_id=ZILXZNB3OTSXMQIGY7SLUVNAL3FOUBTGHVJMFOO4DNQKEN4DM7XA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: read tcp 5.161.89.252:40304->89.149.201.147:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913582424 (632h9m46.61115076s) [noise=true , address=89.149.201.147:4018 , piece_id=SD3AVM66IUR7PBNEI7OC2HGSQEJEROQOB2U7WHCGCIA5BUVP7THA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 5.161.89.252:38616->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913664918 (632h9m46.611233235s) [noise=true , address=89.149.201.147:4009 , piece_id=TBLY5IR3JRD4UHAU2MQA5RARB4VHAZMZLKTXSJXIEQNSDPC32FIA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F, address:89.149.201.147:4009): protocol: expected piece hash; context deadline exceeded; EOF
Dec 31 23:38:58.913721445 (632h9m46.61128979s) [noise=true , address=89.149.201.147:4005 , piece_id=KMWVFGJ563IVNRBSB2KXMZ2MIOLLBXRCPT2LWLC4SJQYSSRIUDOQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 5.161.89.252:30528->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913799671 (632h9m46.611367977s) [noise=true , address=81.171.14.70:4037 , piece_id=OG5GSU4YW3YUHP7EA2QYS3SHAM57JPWPXXPXHYA37EK5NUUQCHAQ, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: read tcp 5.161.89.252:8096->81.171.14.70:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.913927670 (632h9m46.611496005s) [noise=true , address=185.167.204.205:4014 , piece_id=AETJXBY46QEGZW453VBMHVZOOHYIP3V2XN5DM2DCTQMTGUET66MA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:38046->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.914072921 (632h9m46.611641267s) [noise=true , address=89.149.201.147:4001 , piece_id=F6ZB75TOZHPUJRIGJD25NWNWPPOHYL7NADPO3KRVO63KL3QDAXCA, seg_pos={0 0}, upload=556129]: Done uploading piece. err: ecclient: upload failed (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8, address:89.149.201.147:4001): protocol: read tcp 5.161.89.252:12514->89.149.201.147:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:38:58.914121051 (632h9m46.611689397s) [seg_pos={0 0}, upload=556129]: Done waiting for segment. successful: 8 optimal: 26 errs: <nil>
Dec 31 23:38:59.016542255 (632h9m46.714110571s) [upload=556129]: Done uploading segments...
Dec 31 23:38:59.016565057 (632h9m46.714133373s) [upload=556129]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 8)
Duration: 1m0.105961341s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
25.93397901s trace:
https://jaeger.dev.storj.io/trace/682184c7f92ff532
FAIL (2025-12-31 23:37:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 2)
======LOGS======
[noise=true , address=185.167.204.205:4030 , piece_id=BELICAJYJWQAP6VJGDVWNJ27I5QNDT5HJ7UZHZWNVJCEBHWRRKZQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:59174->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651055646 (632h7m53.348623953s) [noise=true , address=185.167.204.205:4007 , piece_id=DNZNUYSK6C2YMD7J36KFD5NQ4VVQZMLW62HMJULSKDKVE6DYD45A, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: read tcp 5.161.89.252:25340->185.167.204.205:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651172545 (632h7m53.348740851s) [noise=true , address=185.167.204.205:4026 , piece_id=LQS3QGD67MLZ5VUPJ2ESUDT5BSM33NM53REOQDUGE7RSMEKXNFEA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: read tcp 5.161.89.252:30086->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651279624 (632h7m53.348847931s) [noise=true , address=185.167.204.206:4049 , piece_id=UJDAWVBI3WVKTBWJHSEZ257L7PLZFBLOI53SFQA4SX457JXN5F7Q, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: read tcp 5.161.89.252:44452->185.167.204.206:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651434664 (632h7m53.349002971s) [noise=true , address=185.167.204.205:4008 , piece_id=MRHKDXSQXAJTLBBPHCS3S7EYI4TP4RO2RKCDBOQM5NKBQQFMULFQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: read tcp 5.161.89.252:51752->185.167.204.205:4008: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651525094 (632h7m53.34909341s) [noise=true , address=89.149.201.147:4007 , piece_id=4P5MM2VX3WSWYASEEEOMWXW24HW5L2QXSZ6EAFXSERSNURPX6GSQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 5.161.89.252:58178->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651644266 (632h7m53.349212582s) [noise=true , address=185.167.204.205:4009 , piece_id=Z2VEOIMGXQEMB2Y5DER5B33KLBMRHIIMZPCV6EPY365L2HHZPDXA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: read tcp 5.161.89.252:10354->185.167.204.205:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651727812 (632h7m53.349296118s) [noise=true , address=81.171.14.70:4040 , piece_id=GKO2TTGWSUDUMC7R5YTHL5SCQ662AUKND6ECEXNDAEOI3YWRZCKA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 5.161.89.252:22418->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651803905 (632h7m53.349372211s) [noise=true , address=81.171.14.70:4035 , piece_id=KPSYZO64M5NP3VTNS6BCFRQRK7FW6TYXCX6NXM7XJKA7HMYBSXJQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:53050->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651879656 (632h7m53.349447972s) [noise=true , address=89.149.201.147:4012 , piece_id=KEDOIJFSRBRKN7EPXWU52VPMM5PR6EBQOPSJC5VKM5Z4LA5OX57Q, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: read tcp 5.161.89.252:39176->89.149.201.147:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.652678669 (632h7m53.350246985s) [noise=true , address=89.149.201.147:4006 , piece_id=7J564H45DXRXO35APWNQWTROOQGNZA7V5SAEZBUDMNAJVHRNO4ZA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12YTf9jvAKd4gVTxTZC6gGLdXC17ygbrqvqoKtmPbtHyc65YjDU, address:89.149.201.147:4006): protocol: read tcp 5.161.89.252:47184->89.149.201.147:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.652798874 (632h7m53.35036719s) [noise=true , address=185.167.204.206:4064 , piece_id=Q2CBZZKY5E2HU5P7DH6PC45LQ3FEJUUX4XDYMYWTUMJDH5ZHYFSQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1ZB3emcQntFxpr7zBiE98eesnjoTnNKinw3FPtGU2TgznGakeb, address:185.167.204.206:4064): protocol: read tcp 5.161.89.252:57986->185.167.204.206:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.652968500 (632h7m53.350536807s) [noise=true , address=81.171.14.70:4036 , piece_id=TEZQDA63CTHOBH43JITJZLQDUQVYZCD3BI2CWJOOCDQ323TYA3XA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:39622->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653156712 (632h7m53.350725018s) [noise=true , address=89.149.201.147:4009 , piece_id=CNOZIM4NQ7PYYWNAHUYLLJD744QWLPCXPAAOWDDZQTREUCJY6EIQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F, address:89.149.201.147:4009): protocol: read tcp 5.161.89.252:51954->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653261939 (632h7m53.350830245s) [noise=true , address=81.171.14.70:4041 , piece_id=IOGEOYDTN5RY3NHUCLBOAKM636KLENRKFTNPBS34JL7P3T3WJALQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 5.161.89.252:25646->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653341978 (632h7m53.350910294s) [noise=true , address=89.149.201.147:4005 , piece_id=GB3CTQKGZNHBNEQ6HQTCI4VZALG6HGY2EJHF7YTT3STEOWV73LGQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 5.161.89.252:27470->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653418641 (632h7m53.350986957s) [noise=true , address=81.171.14.70:4043 , piece_id=UVLOQWK34YHBL52KKDNL65PXFWJUQ3PJCV3IR4ZTXQDINU3Y2ITQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: read tcp 5.161.89.252:13052->81.171.14.70:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653516644 (632h7m53.35108495s) [noise=true , address=89.149.201.147:4008 , piece_id=DEUZT2U4Z7IWCVGJNV7XKL6QGEIQV7MEPWCD67C4KTEZH7TOWWLA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn, address:89.149.201.147:4008): protocol: read tcp 5.161.89.252:38410->89.149.201.147:4008: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653619807 (632h7m53.351188123s) [noise=true , address=81.171.14.70:4031 , piece_id=THJJAZBZOQSO2YIMW66XUIL5CSKS2VRXPLLZCMR33D6ZUHSN6KWQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: read tcp 5.161.89.252:9142->81.171.14.70:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.655204898 (632h7m53.352773214s) [noise=true , address=81.171.14.70:4028 , piece_id=465XS3UF7BBXRZQ2X7CBPGGJHE3NZGYUIERAFPTGGYI3XRLK5HSA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: read tcp 5.161.89.252:54604->81.171.14.70:4028: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 31 23:37:05.655225907 (632h7m53.352794223s) [seg_pos={0 0}, upload=556097]: Done waiting for segment. successful: 2 optimal: 26 errs: <nil>
Dec 31 23:37:05.754284589 (632h7m53.451852925s) [upload=556097]: Done uploading segments...
Dec 31 23:37:05.754316369 (632h7m53.451884715s) [upload=556097]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 2)
Duration: 25.933992756s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
1m0.105919307s trace:
https://jaeger.dev.storj.io/trace/682184c7f92ff532
FAIL (2025-12-31 23:37:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 2)
======LOGS======
[noise=true , address=185.167.204.205:4030 , piece_id=BELICAJYJWQAP6VJGDVWNJ27I5QNDT5HJ7UZHZWNVJCEBHWRRKZQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12w5R2Whxgg4AYNNvTkKmFq4pzahF1uZiVzwgVygc9iZDAXMdSk, address:185.167.204.205:4030): protocol: read tcp 5.161.89.252:59174->185.167.204.205:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651055646 (632h7m53.348623953s) [noise=true , address=185.167.204.205:4007 , piece_id=DNZNUYSK6C2YMD7J36KFD5NQ4VVQZMLW62HMJULSKDKVE6DYD45A, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: read tcp 5.161.89.252:25340->185.167.204.205:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651172545 (632h7m53.348740851s) [noise=true , address=185.167.204.205:4026 , piece_id=LQS3QGD67MLZ5VUPJ2ESUDT5BSM33NM53REOQDUGE7RSMEKXNFEA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: read tcp 5.161.89.252:30086->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651279624 (632h7m53.348847931s) [noise=true , address=185.167.204.206:4049 , piece_id=UJDAWVBI3WVKTBWJHSEZ257L7PLZFBLOI53SFQA4SX457JXN5F7Q, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12VQJie7zan6SRJRmRm36gsVfY8UsYqtJeFRgKZy6m5d6B1aLFM, address:185.167.204.206:4049): protocol: read tcp 5.161.89.252:44452->185.167.204.206:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651434664 (632h7m53.349002971s) [noise=true , address=185.167.204.205:4008 , piece_id=MRHKDXSQXAJTLBBPHCS3S7EYI4TP4RO2RKCDBOQM5NKBQQFMULFQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: read tcp 5.161.89.252:51752->185.167.204.205:4008: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651525094 (632h7m53.34909341s) [noise=true , address=89.149.201.147:4007 , piece_id=4P5MM2VX3WSWYASEEEOMWXW24HW5L2QXSZ6EAFXSERSNURPX6GSQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1Q7d4vj1KoWhDr6xd9gEfFrZ1sV3Yro2AFke2f3Ke1rXsGnvew, address:89.149.201.147:4007): protocol: read tcp 5.161.89.252:58178->89.149.201.147:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651644266 (632h7m53.349212582s) [noise=true , address=185.167.204.205:4009 , piece_id=Z2VEOIMGXQEMB2Y5DER5B33KLBMRHIIMZPCV6EPY365L2HHZPDXA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1bGUwUMQeVuL4HCvsRkMSPsgzByNWwMvx6t7HmQ7WLsXoJytgb, address:185.167.204.205:4009): protocol: read tcp 5.161.89.252:10354->185.167.204.205:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651727812 (632h7m53.349296118s) [noise=true , address=81.171.14.70:4040 , piece_id=GKO2TTGWSUDUMC7R5YTHL5SCQ662AUKND6ECEXNDAEOI3YWRZCKA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1gannW154cmVP6N5BowytKfDorZDRwx2M5pi6akqjZFU38PGjV, address:81.171.14.70:4040): protocol: read tcp 5.161.89.252:22418->81.171.14.70:4040: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651803905 (632h7m53.349372211s) [noise=true , address=81.171.14.70:4035 , piece_id=KPSYZO64M5NP3VTNS6BCFRQRK7FW6TYXCX6NXM7XJKA7HMYBSXJQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:53050->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.651879656 (632h7m53.349447972s) [noise=true , address=89.149.201.147:4012 , piece_id=KEDOIJFSRBRKN7EPXWU52VPMM5PR6EBQOPSJC5VKM5Z4LA5OX57Q, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: read tcp 5.161.89.252:39176->89.149.201.147:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.652678669 (632h7m53.350246985s) [noise=true , address=89.149.201.147:4006 , piece_id=7J564H45DXRXO35APWNQWTROOQGNZA7V5SAEZBUDMNAJVHRNO4ZA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12YTf9jvAKd4gVTxTZC6gGLdXC17ygbrqvqoKtmPbtHyc65YjDU, address:89.149.201.147:4006): protocol: read tcp 5.161.89.252:47184->89.149.201.147:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.652798874 (632h7m53.35036719s) [noise=true , address=185.167.204.206:4064 , piece_id=Q2CBZZKY5E2HU5P7DH6PC45LQ3FEJUUX4XDYMYWTUMJDH5ZHYFSQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1ZB3emcQntFxpr7zBiE98eesnjoTnNKinw3FPtGU2TgznGakeb, address:185.167.204.206:4064): protocol: read tcp 5.161.89.252:57986->185.167.204.206:4064: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.652968500 (632h7m53.350536807s) [noise=true , address=81.171.14.70:4036 , piece_id=TEZQDA63CTHOBH43JITJZLQDUQVYZCD3BI2CWJOOCDQ323TYA3XA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:39622->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653156712 (632h7m53.350725018s) [noise=true , address=89.149.201.147:4009 , piece_id=CNOZIM4NQ7PYYWNAHUYLLJD744QWLPCXPAAOWDDZQTREUCJY6EIQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1sbZWJFMVsppCpbFVfn33bx5uGkhtFpzJm8a2i8DoAfQNsWY5F, address:89.149.201.147:4009): protocol: read tcp 5.161.89.252:51954->89.149.201.147:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653261939 (632h7m53.350830245s) [noise=true , address=81.171.14.70:4041 , piece_id=IOGEOYDTN5RY3NHUCLBOAKM636KLENRKFTNPBS34JL7P3T3WJALQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC, address:81.171.14.70:4041): protocol: read tcp 5.161.89.252:25646->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653341978 (632h7m53.350910294s) [noise=true , address=89.149.201.147:4005 , piece_id=GB3CTQKGZNHBNEQ6HQTCI4VZALG6HGY2EJHF7YTT3STEOWV73LGQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK, address:89.149.201.147:4005): protocol: read tcp 5.161.89.252:27470->89.149.201.147:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653418641 (632h7m53.350986957s) [noise=true , address=81.171.14.70:4043 , piece_id=UVLOQWK34YHBL52KKDNL65PXFWJUQ3PJCV3IR4ZTXQDINU3Y2ITQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12o3f1bhpf5JrqLvdu57xT2JFxNwAzkUT4bW8qdjn5nbJ1mQDko, address:81.171.14.70:4043): protocol: read tcp 5.161.89.252:13052->81.171.14.70:4043: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653516644 (632h7m53.35108495s) [noise=true , address=89.149.201.147:4008 , piece_id=DEUZT2U4Z7IWCVGJNV7XKL6QGEIQV7MEPWCD67C4KTEZH7TOWWLA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:1QN8VP6o7qGw9Dd8EafU9xe7s7Cuf1FyqPMbnRaznrfPxfEUUn, address:89.149.201.147:4008): protocol: read tcp 5.161.89.252:38410->89.149.201.147:4008: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.653619807 (632h7m53.351188123s) [noise=true , address=81.171.14.70:4031 , piece_id=THJJAZBZOQSO2YIMW66XUIL5CSKS2VRXPLLZCMR33D6ZUHSN6KWQ, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: read tcp 5.161.89.252:9142->81.171.14.70:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:37:05.655204898 (632h7m53.352773214s) [noise=true , address=81.171.14.70:4028 , piece_id=465XS3UF7BBXRZQ2X7CBPGGJHE3NZGYUIERAFPTGGYI3XRLK5HSA, seg_pos={0 0}, upload=556097]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: read tcp 5.161.89.252:54604->81.171.14.70:4028: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 31 23:37:05.655225907 (632h7m53.352794223s) [seg_pos={0 0}, upload=556097]: Done waiting for segment. successful: 2 optimal: 26 errs: <nil>
Dec 31 23:37:05.754284589 (632h7m53.451852925s) [upload=556097]: Done uploading segments...
Dec 31 23:37:05.754316369 (632h7m53.451884715s) [upload=556097]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 2)
Duration: 1m0.105936119s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.111357646s trace:
https://jaeger.dev.storj.io/trace/2b4c5070ba074810
FAIL (2025-12-31 23:35:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 10)
======LOGS======
A, seg_pos={0 0}, upload=556084]: Done uploading piece. err: <nil>
Dec 31 23:35:28.294143290 (632h6m15.991711606s) [noise=true , address=81.171.14.70:4025 , piece_id=GEOABKXOSMS6K3LRWFOEIJSZA3CSWNSNNYVZK77V2O5RTBPAQ3QA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: <nil>
Dec 31 23:35:28.341642062 (632h6m16.039210378s) [noise=true , address=185.167.204.206:4045 , piece_id=Q4JIET7MHB6XYMX7F6XIBS5Q5ZAY3WHXUFPDGULYMFVKX4ZSUKTA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: <nil>
Dec 31 23:35:28.659185996 (632h6m16.356754341s) [noise=true , address=81.171.14.70:4034 , piece_id=4FKSPKZ3RNKYGJWKIXSYY4VSU5CMZQRDHM7ILHERI4UVRQQ6VEFA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: <nil>
Dec 31 23:35:30.142646792 (632h6m17.840215098s) [noise=true , address=81.171.14.70:4031 , piece_id=YCAKNZYEMEVN7GXXNTEHGFB7VWY5NLEMU7CC4RWZAF2YQ52PTDKQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: <nil>
Dec 31 23:35:55.468399698 (632h6m43.165968034s) [noise=true , address=89.149.201.147:4013 , piece_id=PLTSFKRLZUYEKBAZW5RRVFCOZVDKZC3XFI4VFPBBXQBDJ2QAZOQA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:125ZKP9g2cY7B3b8eyfHqoChUG2nYtz3nbFRGA6szgtACzT4M6J, address:89.149.201.147:4013): protocol: read tcp 5.161.89.252:44562->89.149.201.147:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.468741146 (632h6m43.166309482s) [noise=true , address=185.167.204.205:4013 , piece_id=MTBHX5WGDDDRIYNXWQRYPTIP5AY4WXZDETX2PR5DOEXDC3EA2C7A, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: read tcp 5.161.89.252:10708->185.167.204.205:4013: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.468882360 (632h6m43.166450696s) [noise=true , address=185.167.204.206:4038 , piece_id=3D2YLGQBOYTCSDOEQNNQ6DVFBXBQCVY4WGDFKJ4BNUWTJQ4YUEWA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: rpc: dial tcp 185.167.204.206:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469005991 (632h6m43.166574327s) [noise=true , address=185.167.204.205:4024 , piece_id=SDJGNGQW5IXPIDHFCZLH767ATIICOWQFKZ2KPIUO52FNNLC2S76Q, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12Wa3aDy81HhCKQiRo9BeZGYZqcVJWbLpN4TxdZaMHuitPSTxLV, address:185.167.204.205:4024): protocol: read tcp 5.161.89.252:47026->185.167.204.205:4024: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469039934 (632h6m43.166608271s) [noise=true , address=81.171.14.70:4030 , piece_id=BP32LCW4IFWLR5IWDVYCFTQNUM7OITUAFDKZGIIMLNDSYRRMDC6Q, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 5.161.89.252:54220->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469201776 (632h6m43.166770112s) [noise=true , address=185.167.204.205:4029 , piece_id=PLICY2KYDU4VK6WV7IWUDCJERSWC4NE2USZQT5NDYTGARYILN24Q, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: read tcp 5.161.89.252:28078->185.167.204.205:4029: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469301333 (632h6m43.166869668s) [noise=true , address=81.171.14.70:4037 , piece_id=GSGCDFDBQZOYB4JPZ7U74JLDLUFDYZDFQAWZRP7VDLJCZBUWA25A, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: read tcp 5.161.89.252:60724->81.171.14.70:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469354942 (632h6m43.166923278s) [noise=true , address=89.149.201.147:4018 , piece_id=5OVDP4IUZAO73GRTEG4G3HB26QLQ5G5C5VACNEHSWR2RJT3MFPPQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1Ku7o12bSmJ1YR3dGetGFcWqs1wJEh53Zkg3H7ELJrt4w1hkua, address:89.149.201.147:4018): protocol: read tcp 5.161.89.252:34212->89.149.201.147:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469476309 (632h6m43.167044646s) [noise=true , address=81.171.14.70:4047 , piece_id=LBRG2QBPBYF7GZGHF5HEX2XKGCL3LFKSDXBEJ7TIUSTAC66GGQRQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: read tcp 5.161.89.252:51266->81.171.14.70:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469487590 (632h6m43.167055926s) [noise=true , address=185.167.204.206:4066 , piece_id=D3BNNTCOVIQ6VGFELEIGJGDNSOVTHPLITU54ROSZIROE5B27MLGA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1ns5eXmL9FCPxzevdEpYg6HZEjHnE3ZRB6JRhY51hMMW2mEmUg, address:185.167.204.206:4066): protocol: read tcp 5.161.89.252:38036->185.167.204.206:4066: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469612063 (632h6m43.167180409s) [noise=true , address=185.167.204.205:4027 , piece_id=FHKKOQ4QE5MX5AS4ZNHLR5UCZV5ZDATTO6B7XRXM2ZDG2DALTAYQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:1CVzMfvFN4cYHWnSVrXBmHjLR9KtGqsMimwVq1QJzjgpbjeszV, address:185.167.204.205:4027): protocol: read tcp 5.161.89.252:59886->185.167.204.205:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469620949 (632h6m43.167189286s) [noise=true , address=89.149.201.147:4022 , piece_id=7JTAMI3XRB3LWJ56Q5ZRR6HCLVTSDKK4O6752INHSH3CBJI5MBMA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:122Vj8XxpS6RnhVweQV3cZWFhj8nrzy3hZ9P7JsuTzEaZcD6iFE, address:89.149.201.147:4022): protocol: read tcp 5.161.89.252:43232->89.149.201.147:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469747376 (632h6m43.167315712s) [noise=true , address=81.171.14.70:4029 , piece_id=AJ6AKB5WNAU5P3AUIMRN7A5BWBEZNQ2OCLA2D245MC6IKNNJBVZQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: read tcp 5.161.89.252:9694->81.171.14.70:4029: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469753177 (632h6m43.167321513s) [noise=true , address=185.167.204.206:4060 , piece_id=LQLFBQABFUBFXF5HBRAJ3KCL4MKECABN73AGMVZXJHI6XS6SUDHQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 5.161.89.252:49426->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469873000 (632h6m43.167441336s) [noise=true , address=185.167.204.206:4039 , piece_id=6YCX3L3EEHUDF7DX3HEZ7BOPPHHDWZRLHJYQXZ4MPYTNKF24OCAA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12FV1y64fPXx14kCr3Tng2sYqXdsxaPeRJXmNVJLCVKQNkCo3Y9, address:185.167.204.206:4039): protocol: read tcp 5.161.89.252:22062->185.167.204.206:4039: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.469878911 (632h6m43.167447257s) [noise=true , address=81.171.14.70:4038 , piece_id=LYMASC6E6D4SUTG3JZ2BIITKJ6ME7ENQLX62IEJ5KTTDOYMEZ2CQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: read tcp 5.161.89.252:37508->81.171.14.70:4038: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.470041936 (632h6m43.167610272s) [noise=true , address=185.167.204.206:4037 , piece_id=UDEI2ALPNNTHVWQPXKCSYAJYKST3ITAV33VBJSVO3UGKOSNJB6GQ, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12nEq1PMWr39K5LBUcViEtEb1KTnsJLjMaBbNGuqpc13WdZb18R, address:185.167.204.206:4037): protocol: read tcp 5.161.89.252:38940->185.167.204.206:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.470128067 (632h6m43.167696403s) [noise=true , address=185.167.204.206:4069 , piece_id=7ASDUIZC6LBKGSPFVROQMEHBECYG6DQLEZZ3NKRHOTW2AQKN2DFA, seg_pos={0 0}, upload=556084]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: read tcp 5.161.89.252:27424->185.167.204.206:4069: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 31 23:35:55.470168001 (632h6m43.167736348s) [seg_pos={0 0}, upload=556084]: Done waiting for segment. successful: 10 optimal: 26 errs: <nil>
Dec 31 23:35:55.578059431 (632h6m43.275627768s) [upload=556084]: Done uploading segments...
Dec 31 23:35:55.578147645 (632h6m43.275715992s) [upload=556084]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 10)
Duration: 1m0.111376952s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
500.965576ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:05:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 500.983921ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
470.721107ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:05:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 470.737708ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
815.685009ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:04:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 815.703543ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
512.046316ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:03:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 512.064681ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
460.053468ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:02:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 460.089976ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
494.816661ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:02:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 494.836148ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
498.531229ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:01:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 498.545105ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
542.114519ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:01:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 542.133042ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
527.896547ms trace:
https://jaeger.dev.storj.io/trace/4ccbadcc638911c5
FAIL (2025-12-31 22:00:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 527.915092ms
core
uplink
us-select-1
us1-placement-20
|