Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 442.340364ms trace: https://jaeger.dev.storj.io/trace/478525f8c956dfe8 FAIL (2026-01-04 06:12:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.359691ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 644.577513ms trace: https://jaeger.dev.storj.io/trace/478525f8c956dfe8 FAIL (2026-01-04 06:11:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 644.592721ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 475.769784ms trace: https://jaeger.dev.storj.io/trace/478525f8c956dfe8 FAIL (2026-01-04 06:11:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 475.810099ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001247761s trace: https://jaeger.dev.storj.io/trace/478525f8c956dfe8 FAIL (2026-01-04 06:10:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001262768s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 11.171922802s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:55:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.171941728s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 11.146872403s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:54:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.146895187s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 11.164410601s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:53:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.164427822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 364.032512ms trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:52:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 364.050866ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.296838765s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:51:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.296859193s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.511052732s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:50:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.51107237s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.60321948s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:50:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.603239617s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 11.183664274s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:49:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.183681646s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.28957268s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:48:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.28958842s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 11.202097208s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:48:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.20211412s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.117628568s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:47:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.117709269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 7.410236373s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:46:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 7.410287449s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 11.126005934s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:46:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.126022816s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 11.097021296s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:45:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.09703387s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 11.184074108s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:44:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.184096341s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 11.271382578s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:43:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.2714051s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 11.139513758s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:42:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.139534346s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 11.168137157s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:42:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.168154218s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 11.182207109s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:41:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.182228479s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.635649742s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:40:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.635673236s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.247660477s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:40:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.247675266s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.690624843s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:39:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.690647727s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 853.954234ms trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:39:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 853.967108ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.638511248s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:38:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.638528289s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 7.272992392s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:37:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 7.273010907s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 11.215171611s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:37:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.215189824s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 11.155614928s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:36:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.155635036s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 11.180387172s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:36:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.18040725s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 11.136822868s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:35:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.13683992s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 11.161308188s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:34:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.161328447s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.339319083s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:33:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.339341326s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.244518389s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:32:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.244532997s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 10.724611896s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:32:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 10.724642202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 11.176951023s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:31:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.176963476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 11.195624079s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:30:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.19564083s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 11.168162144s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:29:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.168178425s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 10.27811683s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:28:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 10.278138119s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.645055292s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:27:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.645070099s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.278544359s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:26:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.278560819s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 11.161365177s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:26:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.16138301s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 11.139324211s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:25:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.139343827s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 11.119814948s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:24:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.119827372s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.229479381s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:24:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.229498918s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 11.214068001s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:23:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.214088249s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.749545261s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:23:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 1.749586127s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 11.17899549s trace: https://jaeger.dev.storj.io/trace/3bc1fc1076a26566 FAIL (2026-01-04 05:23:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ddress=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.114864245 (709h42m43.812432551s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115156882 (709h42m43.812725198s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115459939 (709h42m43.813028255s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115761181 (709h42m43.813329498s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.115938322 (709h42m43.813506638s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116102880 (709h42m43.813671196s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116336796 (709h42m43.813905122s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116504390 (709h42m43.814072716s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.116763233 (709h42m43.814331549s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117232920 (709h42m43.814801247s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.117565762 (709h42m43.815134068s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118017416 (709h42m43.815585722s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118533882 (709h42m43.816102187s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.118996084 (709h42m43.8165644s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Uploading piece... Jan 4 05:11:56.224890965 (709h42m43.922459301s) [upload=624341]: Next returned nil segment Jan 4 05:11:56.224901384 (709h42m43.92246972s) [upload=624341]: Waiting for error group managing segments... Jan 4 05:11:56.235428990 (709h42m43.932997326s) [noise=true , address=45.91.116.12:4245 , piece_id=5A6DTYZZR5P55MPQYKFQNRUT5CA7Z2HF45M6AD22BTCMV6DNDYYQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.238787765 (709h42m43.936356111s) [noise=true , address=45.91.116.10:4092 , piece_id=4CDC2CS5HWIOZPW6NY6IYRJ7TYSEEXBJA2YM3Z6AVEHBCBDXRHZA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.239941090 (709h42m43.937509426s) [noise=true , address=145.79.227.2:4117 , piece_id=NQGGFXHKHFGKPWX5DDVJTSUZVTCUYALYQDYB6LEP2IHUXZEQQT5A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.261190625 (709h42m43.958758951s) [noise=true , address=77.90.31.18:4818 , piece_id=RDWN5ZLKVUMEO77QT2V6JNKCDSZW57JMYNXWJDYE76TSMDH4U27A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.262369247 (709h42m43.959937583s) [noise=true , address=77.90.31.21:4891 , piece_id=7XQTZVL73HIZL73X2KXOH5IZYUEBKP3FZ2ZMQUQLS5DPHT6X5TDA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:56.676090502 (709h42m44.373658818s) [noise=true , address=64.49.13.170:4726 , piece_id=ZI7XQCO7Y2E4XCRDHGKGEQOQ5EGHLWHDFAXPX73CUKNKHE6WFAQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.107697718 (709h42m44.805266023s) [noise=true , address=145.79.227.34:4912 , piece_id=T652F2XW3D7JJDGQ44WOEUZH32YSZBKJFFKYVKJAFXSWTY5JQZQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.108861391 (709h42m44.806429697s) [noise=true , address=145.79.227.35:4495 , piece_id=IEY6YFGFZMYOG6VFOV772OMJDQZPR34552EMVMSBBNROKUCTATFA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.150942943 (709h42m44.848511259s) [noise=true , address=145.79.227.29:4965 , piece_id=B5HC76QVPYU6HNATHH6KVCVS46K7G7MH243JNXNVVT5TXXXBEM6A, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.151205213 (709h42m44.848773529s) [noise=true , address=145.79.227.32:4857 , piece_id=PP2CE4RJJCU7MTHBOJVTWIEYURQK7YSO7IJCRPNL7YWDSOZQI6MA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.211591139 (709h42m44.909159445s) [noise=true , address=145.79.227.33:4876 , piece_id=26OI27NF5SVMYIYIZVZCVQSDJIKVUICHBLOSR5MH7OZR43XZLZCQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.302026560 (709h42m44.999594876s) [noise=true , address=145.79.227.30:4777 , piece_id=M5KZ5IFOXFIW3FDOUHZ73M5KNNAUWC5QRDFHKHSWT5JHIVKYM4DQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.324440501 (709h42m45.022008807s) [noise=true , address=145.79.227.40:4073 , piece_id=D2QDMUNWIBCOS7KE2BTSRDNYMWCWKMUXSJGMIPA26HKNKSRT4QEQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342873184 (709h42m45.0404415s) [noise=true , address=145.79.227.43:4171 , piece_id=6DND46IBHP4CCRN475OFRLE7OFC5QL3LWQSYZCRFNCZUFUMM5ZQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: <nil> Jan 4 05:11:57.342888533 (709h42m45.040456839s) [seg_pos={0 0}, upload=624341]: Segment reached optimal threshold of 14 pieces. Jan 4 05:11:57.343044775 (709h42m45.040613091s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1C8U9EuxPw2xxuH8rkdoSv4nGaoBB7syRzrZYchYhoQYHxjQSD): protocol: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:26022->64.49.13.169:4707: use of closed network connection Jan 4 05:11:57.343051207 (709h42m45.040619523s) [noise=true , address=64.49.13.169:4707 , piece_id=SFEW2YDXXZLOG7OMH7TAPOWMCXTJLLL42UCMB7PHOOTZAT3MVHQQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343086894 (709h42m45.04065521s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RtYD3H9egoiNUbXsxJMWetAHYqjKGtfnGyNLSn6tbugdHaLXi): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343095159 (709h42m45.040663475s) [noise=true , address=145.79.227.49:4352 , piece_id=NW5OXAHZS625QE4OKH4XD7LHMJPGAEYUYI7FUDUB7ASRERW4QLIA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343242394 (709h42m45.040810711s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Wo98vUt1banixdAJnjEcfWZf7Q9uVFkfSwfhtJxRF8TcvYuKJ): protocol: expected piece hash; context canceled; EOF Jan 4 05:11:57.343258645 (709h42m45.040826961s) [noise=true , address=64.49.13.155:4917 , piece_id=Q2EC73GLKL74DMYIL5MYF3QZ75MFFSGJHODS2KVCMOKZKJ5NJSQA, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343365795 (709h42m45.040934101s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:58794->64.49.13.169:4706: use of closed network connection Jan 4 05:11:57.343372477 (709h42m45.040940784s) [noise=true , address=64.49.13.169:4706 , piece_id=NTPNMWFIXU7Y5W35YPJLK2M75WW6WBEBJK54JILGR4CSW4OHAJJQ, seg_pos={0 0}, upload=624341]: Long tail cancellation due to optimal threshold Jan 4 05:11:57.343386093 (709h42m45.040954399s) [seg_pos={0 0}, upload=624341]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jan 4 05:11:57.343440564 (709h42m45.04100888s) [upload=624341]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 4 05:11:57.343444752 (709h42m45.041013068s) [upload=624341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 4 05:11:57.385200385 (709h42m45.082768692s) [upload=624341]: Done uploading segments... Jan 4 05:11:57.385214210 (709h42m45.082782526s) [upload=624341]: Upload finished. err: <nil> Duration: 11.179010989s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)