Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.635844292s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:52:00 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.635860613s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.204895164s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:51:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.204907598s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
4.384397291s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:51:04 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.384414282s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.653236859s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:50:58 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.653254382s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.161439406s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:50:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.161459864s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.617169079s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:50:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.617189216s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.629321266s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:50:12 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.629344019s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
4.475007817s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:50:05 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.475018737s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.609448748s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:49:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.609466671s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.675884242s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:49:16 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.675907766s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
4.512784934s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:49:13 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.512800263s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.244691685s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:49:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.244713105s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
4.525214731s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:48:44 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.525248614s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.207249989s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:48:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.207271397s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.640904641s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:48:03 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.640929058s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
4.453006951s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:47:45 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.453023452s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.693865708s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:47:22 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.693885687s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.199194951s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:47:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.19921598s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
4.461905008s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:46:50 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.461925306s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.621518388s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:46:31 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.621537313s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.592706555s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:46:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.592737241s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
4.471761686s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:46:13 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.471777103s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.724264109s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:46:09 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.7242868s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.17433431s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:45:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.174353687s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
4.834821938s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:45:48 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.834841715s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
4.636473492s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:45:31 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.636493119s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.2021169s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:45:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.202134652s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.698353148s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:45:04 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.698372273s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.594970568s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:44:48 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.594987159s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.155994606s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:44:45 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.156006889s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.61209507s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:44:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.6121279s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
4.337834036s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:44:30 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.337850618s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.687542758s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:43:49 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.687564959s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.610934974s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:43:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.610953369s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
4.646823575s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:43:22 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.646839244s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.605805998s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:43:18 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.605822049s
core
uplink
us1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
4.704759969s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:42:34 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.70477626s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.150803297s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:42:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.150827904s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.707930941s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:42:26 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.707980304s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.255120286s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:41:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.255142167s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
4.415307542s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:41:22 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.415530628s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.678639015s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:41:19 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.67866254s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.736185963s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:41:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.736209888s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
4.413987295s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:40:51 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.414008405s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.584811193s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:40:32 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.584826471s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.194562486s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:40:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.194584418s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
4.550914488s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:40:25 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.550930046s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.740820548s trace:
https://jaeger.dev.storj.io/trace/576e3c37403ea2f0
FAIL (2025-12-10 05:39:36 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 53)
Duration: 1.740836077s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.142662261s trace:
https://jaeger.dev.storj.io/trace/78954e68dc28a6f0
FAIL (2025-12-10 05:39:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 25)
======LOGS======
32FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483286769 (110h6m40.180855105s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.483441889 (110h6m40.181010225s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484188893 (110h6m40.18175724s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484360284 (110h6m40.18192862s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.484899961 (110h6m40.182468308s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485473173 (110h6m40.183041519s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.485588187 (110h6m40.183156534s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486286061 (110h6m40.183854397s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486614144 (110h6m40.18418248s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.486972244 (110h6m40.184540579s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.487764232 (110h6m40.185332569s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:52.488611475 (110h6m40.186179862s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Uploading piece...
Dec 10 05:35:56.532754683 (110h6m44.230323019s) [upload=96631]: Next returned nil segment
Dec 10 05:35:56.532767777 (110h6m44.230336114s) [upload=96631]: Waiting for error group managing segments...
Dec 10 05:36:03.072814336 (110h6m50.770382652s) [noise=true , address=81.171.14.70:4033 , piece_id=A7M7632FFSAZK7S5S6G4J72AVRGUSAVIFNVQG7QKTLTKEWTZPWFQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:10.426278146 (110h6m58.123846453s) [noise=true , address=185.167.204.205:4003 , piece_id=C6QVIOWAWPAFTTEGTU67E5S2G3F2YNEMN7ZOYJHO7V5DZTJNHBHA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:13.352409435 (110h7m1.049977751s) [noise=true , address=89.149.201.147:4005 , piece_id=SZUARGEGZLRFD5GL53ML3PSRE3NX3WJIU2QLIAIWJ5X74FDM546Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:15.264994828 (110h7m2.962563144s) [noise=true , address=89.149.201.147:4014 , piece_id=FUWY5JUHVSRU2T7F6FRLNXPHHG7BDCNRXDCYAO26VEURJLYVAURQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:17.549961624 (110h7m5.24752993s) [noise=true , address=81.171.14.70:4031 , piece_id=3OKFG434PBYJ3SPU6KMJ42JH2ASMSDYLC3SCHPVYLJMH5ONMRBJQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:19.306486817 (110h7m7.004055133s) [noise=true , address=185.167.204.205:4027 , piece_id=6NJYN7KJKTEVYCSNQA6HQGU33NCRYDQ46FJFSG6BIQU4LZLEJNJA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.162049031 (110h7m10.859617337s) [noise=true , address=81.171.14.70:4046 , piece_id=W5XKF4BDKWOEBYXOBRBCBKY5MORF47WA2C2W7QPT5BMICFJZZBWA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.737363060 (110h7m11.434931366s) [noise=true , address=185.167.204.205:4032 , piece_id=ZSZ2K73DHDZYFLWQRMDRBE7HVHDEXWA7K4EUNPGIVXFAHCVUKTLA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:23.786449800 (110h7m11.484018106s) [noise=true , address=81.171.14.70:4030 , piece_id=X7PDS4QDWBSSWIJES2QO6JI4ZP5EB6T6PMEQNWR42M4MTAOUUF3A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.060159042 (110h7m11.757727358s) [noise=true , address=185.167.204.206:4038 , piece_id=CBND4CAULWE26NIGWFT7DSTQ4WJ4UOGPGPTY2O3TC53QLH3E7RPQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.094361490 (110h7m11.791929795s) [noise=true , address=89.149.201.147:4012 , piece_id=JX6YJHUZEJYORE6IKANUGTMQPBYAY22WNCX2GCMAH74UFJE7X46A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:24.798040935 (110h7m12.495609251s) [noise=true , address=81.171.14.70:4038 , piece_id=JV63T6L5WD2VHUJRS74XJX4VUUNGEROPX2GWX6GXKLFSBI46VBLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.378916283 (110h7m13.076484589s) [noise=true , address=89.149.201.147:4021 , piece_id=BEYZFIFW3J5SRMA3TGXM2TL6E6G7NP4Y4ZY7ZG4B7HBZMRW524IA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.827775932 (110h7m13.525344269s) [noise=true , address=185.167.204.206:4064 , piece_id=O4VT3N4CWID6DN7GVVYJ6SFV3UGLKFHDCLV5TNN3ROQ2TDSBTCKQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.877397152 (110h7m13.574965468s) [noise=true , address=185.167.204.205:4029 , piece_id=HY7GATALKCEDP563CEVY53JEM7S2JLZGQI7IVX7YH3PO5MNQKE7Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:25.944028435 (110h7m13.641596771s) [noise=true , address=81.171.14.70:4048 , piece_id=FHKHNB56U67PNJPDJIBOZLTY2PW6NAGTHLBLL5LLIJOPPXBQDQUQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:26.560902894 (110h7m14.2584712s) [noise=true , address=89.149.201.147:4016 , piece_id=AG43SI5LEABCPKL7BAK2K7OJ4TVXYUIPUS7TVYC5NUQNUQQB5NOA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.812203200 (110h7m15.509771516s) [noise=true , address=89.149.201.147:4023 , piece_id=BI6SXMTHJ5KYB3556G4U6UYG5HRNK5EH64LNT42V3AFNARG5O7LA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:27.867169354 (110h7m15.56473767s) [noise=true , address=89.149.201.147:4006 , piece_id=UEKDVE6NDKI4NV7HD4AKUH4HNYVRF3C7SPIN26KDK5NX6J4NBQ5Q, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:29.445827111 (110h7m17.143395446s) [noise=true , address=185.167.204.205:4008 , piece_id=PZIHSKHE4CALB2V3LHWAX5X6BMMQMSMFWM47MIGVH64G2ZLPMRTQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:32.512177560 (110h7m20.209745866s) [noise=true , address=81.171.14.70:4045 , piece_id=4MHEGBEG6BSYE4TAJ7FKO6QAJEAZAPTAZL2CP77GSZ2NWQCUJDIQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:35.825912333 (110h7m23.523480649s) [noise=true , address=81.171.14.70:4027 , piece_id=X46BF2SMXFGBFRWBG3TN45CSU2V66GVOXQAHTR42H2QVOTRA36OA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.223889733 (110h7m28.92145804s) [noise=true , address=185.167.204.205:4002 , piece_id=ZTKK6O7XAOPFFTLH3BNGFDEMW6C74KDYKXFE6SUA3MIARRPYDT4A, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:41.243892176 (110h7m28.941460483s) [noise=true , address=185.167.204.205:4031 , piece_id=EQZNQZ4BMWYSUX5IKBL2MMX5DPQN5XI4QUIKFR6LWIZI6FLFGJLQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:43.488315198 (110h7m31.185883505s) [noise=true , address=81.171.14.70:4040 , piece_id=WGBE4UIBTC7XEAT2QNJIZBTLSEGQH5HT6KSEC3ML6Y5Z7OI4YMVA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: <nil>
Dec 10 05:36:51.974826346 (110h7m39.672394662s) [noise=true , address=89.149.201.147:4017 , piece_id=ESM6BDBHOTQQUZECBEYOFXEZNMPXIGZZYUS7Y6EXFWBZ4WVFTFUA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ, address:89.149.201.147:4017): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974841284 (110h7m39.672409589s) [noise=true , address=185.167.204.205:4004 , piece_id=PA7UKEJIRVVTB4Q6VKPQ7TZ2C2ZAWHIGZKY3RDNWCRDJIFSVKOSQ, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm, address:185.167.204.205:4004): protocol: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55016->185.167.204.205:4004: use of closed network connection
Dec 10 05:36:51.974903630 (110h7m39.672471946s) [noise=true , address=81.171.14.70:4029 , piece_id=JWETD2NVRRRPRGGDYZAYFI3L5OH6AIDVYRPQKFFSBUUNDJ5DKWEA, seg_pos={0 0}, upload=96631]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: expected piece hash; context deadline exceeded; EOF
Dec 10 05:36:51.974922165 (110h7m39.672490481s) [seg_pos={0 0}, upload=96631]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil>
Dec 10 05:36:52.071976396 (110h7m39.769544712s) [upload=96631]: Done uploading segments...
Dec 10 05:36:52.072005610 (110h7m39.769573927s) [upload=96631]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25)
Duration: 2.142674584s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
4.334766177s trace:
https://jaeger.dev.storj.io/trace/90b6829c07b062b
FAIL (2025-12-10 05:39:25 UTC)
orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 54 but got 50)
Duration: 4.334792898s
core
uplink
ap1
|