Orbiter History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.666406772s trace: https://jaeger.dev.storj.io/trace/5193c4a9831f1dc1 FAIL (2026-01-01 10:03:00 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.666428503s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 4.579157767s trace: https://jaeger.dev.storj.io/trace/5193c4a9831f1dc1 FAIL (2026-01-01 10:02:39 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.579181561s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.380645336s trace: https://jaeger.dev.storj.io/trace/5193c4a9831f1dc1 FAIL (2026-01-01 10:02:06 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.380665184s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 4.60930652s trace: https://jaeger.dev.storj.io/trace/5193c4a9831f1dc1 FAIL (2026-01-01 10:00:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.609318041s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m28.68855451s trace: https://jaeger.dev.storj.io/trace/5193c4a9831f1dc1 FAIL (2026-01-01 10:00:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2m28.68857069s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.33743566s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:55:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.337453665s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 7.652626617s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:54:49 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 7.65265066s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 11.12185673s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:53:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.121870126s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.321170651s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:52:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.32119707s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.246833098s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:51:43 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.246857924s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.245994343s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:50:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.246009292s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 870.909134ms trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:50:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 870.925164ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.246401432s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:49:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.246426258s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 11.183566835s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:48:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.183584607s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.577093056s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:47:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.577111622s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 10.315338351s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:46:21 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 10.315362285s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 11.141156439s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:45:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.141170224s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.272480257s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:44:27 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.272505786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.306315021s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:43:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.306338004s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 11.170061492s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:42:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.170079915s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.544123865s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:42:17 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.544139926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.272075802s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:42:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.272094348s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 11.124752547s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:41:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.124773846s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.292512751s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:40:27 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.292533619s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.539348497s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:39:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.539370199s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.710949292s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:38:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.710968146s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.023726077s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:38:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.023746405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 11.201677752s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:37:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.201697669s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 11.134927758s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:36:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.134960499s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.370847144s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:36:19 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.370862793s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.647157882s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:35:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.647200401s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.754694s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:35:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.754713648s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 819.022061ms trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:34:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 819.037871ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.303657696s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:33:59 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.303679197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.241035166s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:33:27 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.241052218s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.221448258s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:32:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.221474697s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.547709557s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:32:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.547728743s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.959776965s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:31:21 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.95979559s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 11.272707247s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:30:58 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.272747852s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 7.69049658s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:30:29 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 7.690521085s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 11.243554236s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:30:11 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.243609007s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 7.634577854s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:29:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 7.634604204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 767.098212ms trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:28:33 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 767.118298ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.345437087s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:27:57 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.345493431s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.239290965s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:27:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.239304801s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.376021237s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:26:52 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.376037287s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.65635863s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:26:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.656378819s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 11.175827095s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:25:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.175850108s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.647308664s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:25:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 1.647331616s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 11.189828029s trace: https://jaeger.dev.storj.io/trace/7423ba250671cbd FAIL (2026-01-01 09:24:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7673s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.228823193 (641h43m10.926391529s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229223031 (641h43m10.926791367s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.229659937 (641h43m10.927228274s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230072067 (641h43m10.927640413s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230506839 (641h43m10.928075176s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.230914060 (641h43m10.928482396s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.231676043 (641h43m10.929244388s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.232313173 (641h43m10.929881519s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.233623291 (641h43m10.931191627s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.234970407 (641h43m10.932538743s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.235943375 (641h43m10.93351171s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.236753539 (641h43m10.934321874s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.237893047 (641h43m10.935461383s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.238855455 (641h43m10.936423791s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.240104217 (641h43m10.937672554s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Uploading piece... Jan 1 09:12:23.347973445 (641h43m11.045541782s) [upload=564558]: Next returned nil segment Jan 1 09:12:23.347986520 (641h43m11.045554866s) [upload=564558]: Waiting for error group managing segments... Jan 1 09:12:23.360482443 (641h43m11.058050769s) [noise=true , address=45.91.116.5:4320 , piece_id=CDLQXG23BHQ3IVWK2FCSBVJMUGIHYCU46QDCAVTGHIQO5US6NOKA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.366766302 (641h43m11.064334628s) [noise=true , address=77.90.31.21:4890 , piece_id=BMFRDC32GD3VPM4MKW4O5EM4YXXDJMCGHO66IVMEKVPHD33UAZNA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.368835328 (641h43m11.066403654s) [noise=true , address=77.90.31.19:4871 , piece_id=2TKPIFKUVQORPCLP6E2PZPO2KZHNLNEZC46ATUS4SVCEITGF2H6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376052029 (641h43m11.073620365s) [noise=true , address=145.79.227.34:4924 , piece_id=GB662FQNJW5RITZMPXWIR7CEBTDVDDZHYIU4TR5WVPNLXX45YNNQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.376352801 (641h43m11.073921148s) [noise=true , address=77.90.31.12:4600 , piece_id=6Y6CDWNKYNZDX3F742NFN3ZZWPBU6DYDYZ63DCQCC6CZO5PH3N6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.380006446 (641h43m11.077574782s) [noise=true , address=45.91.116.7:4023 , piece_id=TFIDFTFJKLIG42WVKWVUS4LEPL2EVY44UXVV5XRH6OR3F24VAZKQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.382791259 (641h43m11.080359585s) [noise=true , address=77.90.31.10:4535 , piece_id=3TTFBUIF3JZ5U2XM3RNGNGWSFTOP67DOSYIWSOH547HYTRYTXILQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.384255944 (641h43m11.081824281s) [noise=true , address=77.90.31.11:4584 , piece_id=NDO6G5HBHC34HNFVEQBHOSMU3M3FE2I7HX4DOU2323LZEUDWYK6A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.408697493 (641h43m11.106265829s) [noise=true , address=145.79.227.29:4958 , piece_id=KCMJDACUTYHAMTXY5SBPOB4UK2FZMR64DKEMCR54K5QVPKOGN7PA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.440358499 (641h43m11.137926836s) [noise=true , address=145.79.227.35:4508 , piece_id=XHGH3TJYENV5PAIPC5JZQO75WGPHIFLCBRM3YGPV2ZN44IP5C64A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:23.838636073 (641h43m11.53620441s) [noise=true , address=64.49.13.170:4725 , piece_id=HFHFBGX3XF552QUBTHTHRHXDEF46XSYIMGVTDK5IPP4ZOGLZDH4A, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.250193406 (641h43m11.947761742s) [noise=true , address=145.79.227.44:4212 , piece_id=FN6TXVFBNBUJLWDNSGDY7TZ3R22X3WYWWDBJU3OWH2FLSNNYVSVQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.284772531 (641h43m11.982340878s) [noise=true , address=145.79.227.30:4778 , piece_id=H3RRCSB6XTCIAA4Z24XUC66IAOR2ULL5UXLKZ4XRCZTL7QEQKOYA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307645411 (641h43m12.005213747s) [noise=true , address=145.79.227.43:4167 , piece_id=4NUTUYFT6GNH6QO6R3PCSNHKWBICOW2OQGLUD7YUUTV3UBCILO7Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: <nil> Jan 1 09:12:24.307664545 (641h43m12.005232882s) [seg_pos={0 0}, upload=564558]: Segment reached optimal threshold of 14 pieces. Jan 1 09:12:24.307973042 (641h43m12.005541388s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1w21xeA7emhjphgYw2Qyz3uuHgPezuivmVD9KmrKmNZk3TnaGN): protocol: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:32102->64.49.13.169:4716: use of closed network connection Jan 1 09:12:24.307984343 (641h43m12.00555268s) [noise=true , address=64.49.13.169:4716 , piece_id=WIEHGHWVJYDBXXH5SBQHO4OWE3TYFPUGSWIEMOTZLCOA23NHMJ2Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308071798 (641h43m12.005640133s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C4Xom8WT5TZRiX3ZFyRT9tSoAUUFyPNS1VVzN7gT3oDVGDGhK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308089229 (641h43m12.005657565s) [noise=true , address=145.79.227.49:4367 , piece_id=RRZRLXP3QWZUFCJLG374KX2HPYBP3EWQLF3Y2GO3DU4DPUQTSG6Q, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308248627 (641h43m12.005816963s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308258255 (641h43m12.005826591s) [noise=true , address=145.79.227.40:4073 , piece_id=MOLBCHDQYKNI6MQZFXPGY773ZJAYSZQ7GXZTJ2OKS73EEOTURJEQ, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308560900 (641h43m12.006129247s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Jan 1 09:12:24.308575699 (641h43m12.006144034s) [noise=true , address=145.79.227.33:4874 , piece_id=FCXNBNJ6O4QJHXLRNYTEMWAYOV67YR3NXM4UWF47GN7Z5X3MC3YA, seg_pos={0 0}, upload=564558]: Long tail cancellation due to optimal threshold Jan 1 09:12:24.308606416 (641h43m12.006174752s) [seg_pos={0 0}, upload=564558]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 1 09:12:24.308692145 (641h43m12.006260482s) [upload=564558]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 1 09:12:24.308697956 (641h43m12.006266293s) [upload=564558]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 1 09:12:24.342871296 (641h43m12.040439642s) [upload=564558]: Done uploading segments... Jan 1 09:12:24.342888177 (641h43m12.040456514s) [upload=564558]: Upload finished. err: <nil> Duration: 11.189847335s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)