Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 125012 reports (Page 242 of 2501)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.178843531s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:21:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.178864711s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.577120025s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:21:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.577138831s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.380010759s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:21:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.380030285s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 11.236833117s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:20:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.236849348s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 11.165432384s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:19:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.165450077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 11.159616389s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:19:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.15962719s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 11.186966017s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:18:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.186984762s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 11.156944354s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:16:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.156965364s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 11.184875521s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:16:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.184899184s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 11.183390406s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:15:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.18341384s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 11.195510888s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:14:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.195531196s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 11.169228514s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:13:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.169245254s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 11.162406067s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:13:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.162430153s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001567666s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:12:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1m0.001581552s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 454.645278ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:57:19 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.665275ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 440.762359ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:56:32 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 440.784077ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 597.912926ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:56:10 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 597.938302ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 917.538151ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:55:18 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 917.557508ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 496.053203ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:54:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 496.072136ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 598.545589ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:53:49 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 598.563052ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 442.076818ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:53:01 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.094681ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 503.152623ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:51:51 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 503.199401ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 490.716481ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:51:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 490.93596ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 653.184442ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:50:13 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 653.196924ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 474.559769ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:49:27 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 474.578184ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 601.661328ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:48:58 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 601.681446ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 467.470363ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:48:12 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 467.5121ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 489.178728ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:47:53 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 489.204425ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 466.642015ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:47:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 466.658156ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............X............................................] duration: 490.015801ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:47:14 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 490.058452ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................X...........................................] duration: 508.264002ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:46:50 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 508.310759ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................X..........................................] duration: 470.317423ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:45:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 470.334886ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................X.........................................] duration: 496.620146ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:45:32 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 496.635113ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 472.757259ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:44:39 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 472.775484ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.787966018s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:43:53 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.787986605s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 469.806179ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:43:26 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 469.826917ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 4.515139257s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:42:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.515155496s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 449.239987ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:42:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 449.261747ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 487.108368ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:42:14 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 487.122094ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 4.677523158s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:41:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.677543666s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 459.159949ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:41:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.18202ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 4.354191264s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:41:18 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.354208547s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 4.484016223s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:40:29 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.484033515s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 484.085493ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:40:18 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 484.115148ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 4.550806775s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:40:10 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.550828054s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 506.889384ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:39:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 506.918969ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 4.499523123s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:39:46 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.499548611s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 4.472092619s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:39:16 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.472108919s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 4.689013547s trace: https://jaeger.dev.storj.io/trace/72f14475ddd90ea1 FAIL (2026-01-04 03:38:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.689048071s core uplink ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 445.300556ms trace: https://jaeger.dev.storj.io/trace/3b19edbcc6776921 FAIL (2026-01-04 03:38:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 445.316146ms core uplink us-select-1 us1-placement-20

1
...
240
241
242
243
244
...
2501