Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................X.XX.X....X.X......................] duration: 981.843828ms FAIL (2025-08-07 23:37:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 981.910463ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.143741854s FAIL (2025-08-07 23:37:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.143768364s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.............................XXXXX..X.......................] duration: 1.941575593s FAIL (2025-08-07 23:37:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.941593906s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [........................XXXXX.XX............................] duration: 199.585313ms FAIL (2025-08-07 23:37:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.629132566 (6h43m50.216188797s) [upload=6906]: Upload finished. err: <nil> Aug 7 23:11:05.687805040 (6h44m19.274861281s) [upload=6916]: Starting upload Aug 7 23:11:05.687992249 (6h44m19.27504857s) [upload=6916]: Done starting upload Aug 7 23:11:05.688099038 (6h44m19.275155279s) [upload=6916]: Uploading segments... Aug 7 23:11:05.688157528 (6h44m19.275213799s) [upload=6916]: Got next segment. Inline: true Aug 7 23:11:05.688166765 (6h44m19.275223017s) [upload=6916]: Waiting for error group managing segments... Aug 7 23:11:05.688260139 (6h44m19.27531638s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:05.688269687 (6h44m19.275325969s) [upload=6916]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:05.688275919 (6h44m19.27533216s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:05.740655239 (6h44m19.327711451s) [upload=6916]: Done uploading segments... Aug 7 23:11:05.740669697 (6h44m19.327725918s) [upload=6916]: Upload finished. err: <nil> Aug 7 23:12:08.089702100 (6h45m21.67675834s) [upload=6934]: Starting upload Aug 7 23:12:08.089804582 (6h45m21.676860822s) [upload=6934]: Done starting upload Aug 7 23:12:08.089877487 (6h45m21.676933729s) [upload=6934]: Uploading segments... Aug 7 23:12:08.089928542 (6h45m21.676984783s) [upload=6934]: Got next segment. Inline: true Aug 7 23:12:08.089934242 (6h45m21.676990484s) [upload=6934]: Waiting for error group managing segments... Aug 7 23:12:08.089980470 (6h45m21.67703671s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:08.089986311 (6h45m21.677042551s) [upload=6934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:08.089990338 (6h45m21.677046579s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:08.195718921 (6h45m21.782775162s) [upload=6934]: Done uploading segments... Aug 7 23:12:08.195734549 (6h45m21.78279079s) [upload=6934]: Upload finished. err: <nil> Aug 7 23:12:48.910499617 (6h46m2.497555858s) [upload=6943]: Starting upload Aug 7 23:12:48.910615834 (6h46m2.497672085s) [upload=6943]: Done starting upload Aug 7 23:12:48.910681648 (6h46m2.497737898s) [upload=6943]: Uploading segments... Aug 7 23:12:48.910722363 (6h46m2.497778604s) [upload=6943]: Got next segment. Inline: true Aug 7 23:12:48.910727974 (6h46m2.497784214s) [upload=6943]: Waiting for error group managing segments... Aug 7 23:12:48.910775202 (6h46m2.497831443s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:48.910780953 (6h46m2.497837204s) [upload=6943]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:48.910784499 (6h46m2.497840741s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.893357638 (6h46m32.480413889s) [upload=6943]: Done uploading segments... Aug 7 23:13:18.893388065 (6h46m32.480444306s) [upload=6943]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:18.972282939 (6h46m32.559339181s) [upload=6953]: Starting upload Aug 7 23:13:18.972362728 (6h46m32.55941897s) [upload=6953]: Done starting upload Aug 7 23:13:18.972440393 (6h46m32.559496634s) [upload=6953]: Uploading segments... Aug 7 23:13:18.972481941 (6h46m32.559538181s) [upload=6953]: Got next segment. Inline: true Aug 7 23:13:18.972487681 (6h46m32.559543923s) [upload=6953]: Waiting for error group managing segments... Aug 7 23:13:18.972531043 (6h46m32.559587284s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.972536602 (6h46m32.559592843s) [upload=6953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.972540159 (6h46m32.5595964s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:48.959236568 (6h47m2.546292809s) [upload=6953]: Done uploading segments... Aug 7 23:13:48.959259700 (6h47m2.546315941s) [upload=6953]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:04.749315126 (6h47m18.336371367s) [upload=6964]: Starting upload Aug 7 23:14:04.749394674 (6h47m18.336450915s) [upload=6964]: Done starting upload Aug 7 23:14:04.749474242 (6h47m18.336530484s) [upload=6964]: Uploading segments... Aug 7 23:14:04.749532461 (6h47m18.336588702s) [upload=6964]: Got next segment. Inline: true Aug 7 23:14:04.749538312 (6h47m18.336594553s) [upload=6964]: Waiting for error group managing segments... Aug 7 23:14:04.749584738 (6h47m18.33664098s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:04.749590058 (6h47m18.3366463s) [upload=6964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:04.749593755 (6h47m18.336649997s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:34.737068324 (6h47m48.324124536s) [upload=6964]: Done uploading segments... Aug 7 23:14:34.737089104 (6h47m48.324145314s) [upload=6964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:37.277141003 (6h47m50.864197244s) [upload=6974]: Starting upload Aug 7 23:14:37.277320197 (6h47m50.864376439s) [upload=6974]: Done starting upload Aug 7 23:14:37.277421095 (6h47m50.864477337s) [upload=6974]: Uploading segments... Aug 7 23:14:37.277519849 (6h47m50.864576091s) [upload=6974]: Got next segment. Inline: true Aug 7 23:14:37.277527635 (6h47m50.864583875s) [upload=6974]: Waiting for error group managing segments... Aug 7 23:14:37.277578299 (6h47m50.86463454s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:37.277584181 (6h47m50.864640421s) [upload=6974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:37.277587807 (6h47m50.864644049s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:07.263997045 (6h48m20.851053287s) [upload=6974]: Done uploading segments... Aug 7 23:15:07.264036289 (6h48m20.85109253s) [upload=6974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:40.767391975 (6h48m54.354448226s) [upload=6993]: Starting upload Aug 7 23:15:40.767481722 (6h48m54.354537963s) [upload=6993]: Done starting upload Aug 7 23:15:40.767544320 (6h48m54.35460057s) [upload=6993]: Uploading segments... Aug 7 23:15:40.767585375 (6h48m54.354641617s) [upload=6993]: Got next segment. Inline: true Aug 7 23:15:40.767591397 (6h48m54.354647638s) [upload=6993]: Waiting for error group managing segments... Aug 7 23:15:40.767637002 (6h48m54.354693244s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:40.767642423 (6h48m54.354698664s) [upload=6993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:40.767646530 (6h48m54.354702782s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:10.754586120 (6h49m24.341642341s) [upload=6993]: Done uploading segments... Aug 7 23:16:10.754601419 (6h49m24.34165763s) [upload=6993]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:26.719060301 (6h49m40.306116542s) [upload=7005]: Starting upload Aug 7 23:16:26.719141191 (6h49m40.306197433s) [upload=7005]: Done starting upload Aug 7 23:16:26.719210051 (6h49m40.306266292s) [upload=7005]: Uploading segments... Aug 7 23:16:26.719285351 (6h49m40.306341592s) [upload=7005]: Got next segment. Inline: true Aug 7 23:16:26.719292685 (6h49m40.306348926s) [upload=7005]: Waiting for error group managing segments... Aug 7 23:16:26.719340825 (6h49m40.306397066s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:26.719347277 (6h49m40.306403518s) [upload=7005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:26.719351064 (6h49m40.306407305s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:26.802515608 (6h49m40.389571849s) [upload=7005]: Done uploading segments... Aug 7 23:16:26.802535986 (6h49m40.389592227s) [upload=7005]: Upload finished. err: <nil> Aug 7 23:16:42.191377275 (6h49m55.778433516s) [upload=7011]: Starting upload Aug 7 23:16:42.191466421 (6h49m55.778522662s) [upload=7011]: Done starting upload Aug 7 23:16:42.191531582 (6h49m55.778587803s) [upload=7011]: Uploading segments... Aug 7 23:16:42.191607003 (6h49m55.778663224s) [upload=7011]: Got next segment. Inline: true Aug 7 23:16:42.191612414 (6h49m55.778668635s) [upload=7011]: Waiting for error group managing segments... Aug 7 23:16:42.191651346 (6h49m55.778707556s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:42.191656575 (6h49m55.778712787s) [upload=7011]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:42.191659481 (6h49m55.778715702s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:12.173345160 (6h50m25.760401401s) [upload=7011]: Done uploading segments... Aug 7 23:17:12.173364576 (6h50m25.760420816s) [upload=7011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:12.261509015 (6h50m25.848565257s) [upload=7016]: Starting upload Aug 7 23:17:12.261597881 (6h50m25.848654122s) [upload=7016]: Done starting upload Aug 7 23:17:12.261665168 (6h50m25.848721419s) [upload=7016]: Uploading segments... Aug 7 23:17:12.261708137 (6h50m25.848764388s) [upload=7016]: Got next segment. Inline: true Aug 7 23:17:12.261713487 (6h50m25.848769728s) [upload=7016]: Waiting for error group managing segments... Aug 7 23:17:12.261755746 (6h50m25.848811988s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:12.261761196 (6h50m25.848817448s) [upload=7016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:12.261764593 (6h50m25.848820834s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:42.241895867 (6h50m55.828952078s) [upload=7016]: Done uploading segments... Aug 7 23:17:42.241913931 (6h50m55.828970141s) [upload=7016]: Upload finished. err: metaclient: context deadline exceeded Duration: 199.609999ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [........................XXXXXX.X............................] duration: 373.052474ms FAIL (2025-08-07 23:37:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 373.078904ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................X....XXX.X......................] duration: 569.914646ms FAIL (2025-08-07 23:37:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 569.938752ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.............................X.XXXXX..X.....................] duration: 233.659757ms FAIL (2025-08-07 23:37:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 233.684451ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [..........................XXXX.X..X..X......................] duration: 212.463593ms FAIL (2025-08-07 23:37:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 212.484692ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................XXX.X.X.......................] duration: 940.878633ms FAIL (2025-08-07 23:37:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== A5NM4A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.091791202 (6h51m59.678847453s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092306323 (6h51m59.679362574s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092699598 (6h51m59.679755839s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093097069 (6h51m59.68015331s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093493069 (6h51m59.68054931s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093847771 (6h51m59.680904022s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094203605 (6h51m59.681259855s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094597400 (6h51m59.681653641s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094954427 (6h51m59.682010668s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095373879 (6h51m59.68243012s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095742858 (6h51m59.6827991s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096084166 (6h51m59.683140377s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096399684 (6h51m59.683455905s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096721615 (6h51m59.683777826s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097044478 (6h51m59.684100699s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097416412 (6h51m59.684472633s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.172312403 (6h51m59.759368613s) [upload=7040]: Next returned nil segment Aug 7 23:18:46.172323192 (6h51m59.759379403s) [upload=7040]: Waiting for error group managing segments... Aug 7 23:18:46.182690655 (6h51m59.769746865s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.188105241 (6h51m59.775161462s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.196654078 (6h51m59.783710289s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198492931 (6h51m59.785549152s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198779515 (6h51m59.785835726s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.204927762 (6h51m59.791983983s) [noise=true , address=104.36.82.52:4099 , piece_id=H2WVX26COCLZ7KH6AJLYBN4H6JRIUF5IXYED6DRSCINOVGA5NM4A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.208251176 (6h51m59.795307387s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225240327 (6h51m59.812296549s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225308985 (6h51m59.812365196s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.245212838 (6h51m59.832269059s) [noise=true , address=104.36.82.4:4818 , piece_id=U5SIEBGYEKIRZ4FD46ZT3L3AVH7MIVC3JRKPC7V46F6MGFT5QFXQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.262860569 (6h51m59.84991678s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.287460286 (6h51m59.874516498s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.323021195 (6h51m59.910077436s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337567646 (6h51m59.924623857s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337650712 (6h51m59.924706923s) [seg_pos={0 0}, upload=7040]: Segment reached optimal threshold of 14 pieces. Aug 7 23:18:46.339154849 (6h51m59.92621106s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Lp3H6533e4oUt4mbiWJMjLRi5fb2wQr7RrntSpRm9xqF1TcEi): protocol: rpc: dial tcp 104.36.82.61:4379: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339496567 (6h51m59.926552778s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kNK8y5et7MiiWNDT6iaC8YBAk3PX6WX1XSuU4QMSFBp3KymBb): protocol: read tcp 5.161.89.252:19812->64.49.13.154:4930: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339638873 (6h51m59.926695084s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rUdrYUJgCF8aa3MdNe82dT765nqW46by7mJ2phMSpfHAhzgU2): protocol: read tcp 5.161.89.252:12938->104.36.82.46:4512: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339695828 (6h51m59.92675204s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ckJ4ZkPjcb4ZiJ2EauhxsyBMNvGWaVeJxvmiFGYhbxiaLT5uy): protocol: read tcp 5.161.89.252:28596->64.49.13.155:4922: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339725374 (6h51m59.926781605s) [seg_pos={0 0}, upload=7040]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:18:46.339804101 (6h51m59.926860312s) [upload=7040]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:46.339836361 (6h51m59.926892572s) [upload=7040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:46.393522662 (6h51m59.980578873s) [upload=7040]: Done uploading segments... Aug 7 23:18:46.393540715 (6h51m59.980596927s) [upload=7040]: Upload finished. err: <nil> Aug 7 23:20:00.141045360 (6h53m13.728101581s) [upload=7062]: Starting upload Aug 7 23:20:00.141098730 (6h53m13.728154951s) [upload=7062]: Done starting upload Aug 7 23:20:00.141249572 (6h53m13.728305793s) [upload=7062]: Uploading segments... Aug 7 23:20:00.141576813 (6h53m13.728633034s) [upload=7062]: Got next segment. Inline: false Aug 7 23:20:00.141621376 (6h53m13.728677597s) [upload=7062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:00.141625283 (6h53m13.728681494s) [upload=7062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:21:00.122447463 (6h54m13.709503674s) [upload=7062]: Next segment err: context deadline exceeded Aug 7 23:21:00.122599688 (6h54m13.709655909s) [upload=7062]: Done uploading segments... Aug 7 23:21:00.122624775 (6h54m13.709680986s) [upload=7062]: Upload finished. err: context deadline exceeded Duration: 940.90876ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.........................XXXXX.XX...........................] duration: 191.031457ms FAIL (2025-08-07 23:37:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.629132566 (6h43m50.216188797s) [upload=6906]: Upload finished. err: <nil> Aug 7 23:11:05.687805040 (6h44m19.274861281s) [upload=6916]: Starting upload Aug 7 23:11:05.687992249 (6h44m19.27504857s) [upload=6916]: Done starting upload Aug 7 23:11:05.688099038 (6h44m19.275155279s) [upload=6916]: Uploading segments... Aug 7 23:11:05.688157528 (6h44m19.275213799s) [upload=6916]: Got next segment. Inline: true Aug 7 23:11:05.688166765 (6h44m19.275223017s) [upload=6916]: Waiting for error group managing segments... Aug 7 23:11:05.688260139 (6h44m19.27531638s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:05.688269687 (6h44m19.275325969s) [upload=6916]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:05.688275919 (6h44m19.27533216s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:05.740655239 (6h44m19.327711451s) [upload=6916]: Done uploading segments... Aug 7 23:11:05.740669697 (6h44m19.327725918s) [upload=6916]: Upload finished. err: <nil> Aug 7 23:12:08.089702100 (6h45m21.67675834s) [upload=6934]: Starting upload Aug 7 23:12:08.089804582 (6h45m21.676860822s) [upload=6934]: Done starting upload Aug 7 23:12:08.089877487 (6h45m21.676933729s) [upload=6934]: Uploading segments... Aug 7 23:12:08.089928542 (6h45m21.676984783s) [upload=6934]: Got next segment. Inline: true Aug 7 23:12:08.089934242 (6h45m21.676990484s) [upload=6934]: Waiting for error group managing segments... Aug 7 23:12:08.089980470 (6h45m21.67703671s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:08.089986311 (6h45m21.677042551s) [upload=6934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:08.089990338 (6h45m21.677046579s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:08.195718921 (6h45m21.782775162s) [upload=6934]: Done uploading segments... Aug 7 23:12:08.195734549 (6h45m21.78279079s) [upload=6934]: Upload finished. err: <nil> Aug 7 23:12:48.910499617 (6h46m2.497555858s) [upload=6943]: Starting upload Aug 7 23:12:48.910615834 (6h46m2.497672085s) [upload=6943]: Done starting upload Aug 7 23:12:48.910681648 (6h46m2.497737898s) [upload=6943]: Uploading segments... Aug 7 23:12:48.910722363 (6h46m2.497778604s) [upload=6943]: Got next segment. Inline: true Aug 7 23:12:48.910727974 (6h46m2.497784214s) [upload=6943]: Waiting for error group managing segments... Aug 7 23:12:48.910775202 (6h46m2.497831443s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:48.910780953 (6h46m2.497837204s) [upload=6943]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:48.910784499 (6h46m2.497840741s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.893357638 (6h46m32.480413889s) [upload=6943]: Done uploading segments... Aug 7 23:13:18.893388065 (6h46m32.480444306s) [upload=6943]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:18.972282939 (6h46m32.559339181s) [upload=6953]: Starting upload Aug 7 23:13:18.972362728 (6h46m32.55941897s) [upload=6953]: Done starting upload Aug 7 23:13:18.972440393 (6h46m32.559496634s) [upload=6953]: Uploading segments... Aug 7 23:13:18.972481941 (6h46m32.559538181s) [upload=6953]: Got next segment. Inline: true Aug 7 23:13:18.972487681 (6h46m32.559543923s) [upload=6953]: Waiting for error group managing segments... Aug 7 23:13:18.972531043 (6h46m32.559587284s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.972536602 (6h46m32.559592843s) [upload=6953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.972540159 (6h46m32.5595964s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:48.959236568 (6h47m2.546292809s) [upload=6953]: Done uploading segments... Aug 7 23:13:48.959259700 (6h47m2.546315941s) [upload=6953]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:04.749315126 (6h47m18.336371367s) [upload=6964]: Starting upload Aug 7 23:14:04.749394674 (6h47m18.336450915s) [upload=6964]: Done starting upload Aug 7 23:14:04.749474242 (6h47m18.336530484s) [upload=6964]: Uploading segments... Aug 7 23:14:04.749532461 (6h47m18.336588702s) [upload=6964]: Got next segment. Inline: true Aug 7 23:14:04.749538312 (6h47m18.336594553s) [upload=6964]: Waiting for error group managing segments... Aug 7 23:14:04.749584738 (6h47m18.33664098s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:04.749590058 (6h47m18.3366463s) [upload=6964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:04.749593755 (6h47m18.336649997s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:34.737068324 (6h47m48.324124536s) [upload=6964]: Done uploading segments... Aug 7 23:14:34.737089104 (6h47m48.324145314s) [upload=6964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:37.277141003 (6h47m50.864197244s) [upload=6974]: Starting upload Aug 7 23:14:37.277320197 (6h47m50.864376439s) [upload=6974]: Done starting upload Aug 7 23:14:37.277421095 (6h47m50.864477337s) [upload=6974]: Uploading segments... Aug 7 23:14:37.277519849 (6h47m50.864576091s) [upload=6974]: Got next segment. Inline: true Aug 7 23:14:37.277527635 (6h47m50.864583875s) [upload=6974]: Waiting for error group managing segments... Aug 7 23:14:37.277578299 (6h47m50.86463454s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:37.277584181 (6h47m50.864640421s) [upload=6974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:37.277587807 (6h47m50.864644049s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:07.263997045 (6h48m20.851053287s) [upload=6974]: Done uploading segments... Aug 7 23:15:07.264036289 (6h48m20.85109253s) [upload=6974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:40.767391975 (6h48m54.354448226s) [upload=6993]: Starting upload Aug 7 23:15:40.767481722 (6h48m54.354537963s) [upload=6993]: Done starting upload Aug 7 23:15:40.767544320 (6h48m54.35460057s) [upload=6993]: Uploading segments... Aug 7 23:15:40.767585375 (6h48m54.354641617s) [upload=6993]: Got next segment. Inline: true Aug 7 23:15:40.767591397 (6h48m54.354647638s) [upload=6993]: Waiting for error group managing segments... Aug 7 23:15:40.767637002 (6h48m54.354693244s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:40.767642423 (6h48m54.354698664s) [upload=6993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:40.767646530 (6h48m54.354702782s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:10.754586120 (6h49m24.341642341s) [upload=6993]: Done uploading segments... Aug 7 23:16:10.754601419 (6h49m24.34165763s) [upload=6993]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:26.719060301 (6h49m40.306116542s) [upload=7005]: Starting upload Aug 7 23:16:26.719141191 (6h49m40.306197433s) [upload=7005]: Done starting upload Aug 7 23:16:26.719210051 (6h49m40.306266292s) [upload=7005]: Uploading segments... Aug 7 23:16:26.719285351 (6h49m40.306341592s) [upload=7005]: Got next segment. Inline: true Aug 7 23:16:26.719292685 (6h49m40.306348926s) [upload=7005]: Waiting for error group managing segments... Aug 7 23:16:26.719340825 (6h49m40.306397066s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:26.719347277 (6h49m40.306403518s) [upload=7005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:26.719351064 (6h49m40.306407305s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:26.802515608 (6h49m40.389571849s) [upload=7005]: Done uploading segments... Aug 7 23:16:26.802535986 (6h49m40.389592227s) [upload=7005]: Upload finished. err: <nil> Aug 7 23:16:42.191377275 (6h49m55.778433516s) [upload=7011]: Starting upload Aug 7 23:16:42.191466421 (6h49m55.778522662s) [upload=7011]: Done starting upload Aug 7 23:16:42.191531582 (6h49m55.778587803s) [upload=7011]: Uploading segments... Aug 7 23:16:42.191607003 (6h49m55.778663224s) [upload=7011]: Got next segment. Inline: true Aug 7 23:16:42.191612414 (6h49m55.778668635s) [upload=7011]: Waiting for error group managing segments... Aug 7 23:16:42.191651346 (6h49m55.778707556s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:42.191656575 (6h49m55.778712787s) [upload=7011]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:42.191659481 (6h49m55.778715702s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:12.173345160 (6h50m25.760401401s) [upload=7011]: Done uploading segments... Aug 7 23:17:12.173364576 (6h50m25.760420816s) [upload=7011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:12.261509015 (6h50m25.848565257s) [upload=7016]: Starting upload Aug 7 23:17:12.261597881 (6h50m25.848654122s) [upload=7016]: Done starting upload Aug 7 23:17:12.261665168 (6h50m25.848721419s) [upload=7016]: Uploading segments... Aug 7 23:17:12.261708137 (6h50m25.848764388s) [upload=7016]: Got next segment. Inline: true Aug 7 23:17:12.261713487 (6h50m25.848769728s) [upload=7016]: Waiting for error group managing segments... Aug 7 23:17:12.261755746 (6h50m25.848811988s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:12.261761196 (6h50m25.848817448s) [upload=7016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:12.261764593 (6h50m25.848820834s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:42.241895867 (6h50m55.828952078s) [upload=7016]: Done uploading segments... Aug 7 23:17:42.241913931 (6h50m55.828970141s) [upload=7016]: Upload finished. err: metaclient: context deadline exceeded Duration: 191.050264ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................X.XX.X....X.X.....................] duration: 475.752023ms FAIL (2025-08-07 23:37:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 475.848064ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 200.274008ms FAIL (2025-08-07 23:37:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 200.292944ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........................XXXXXX.X.X.........................] duration: 1.776335215s FAIL (2025-08-07 23:37:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.776357637s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.........................XXXXXX.X...........................] duration: 428.569323ms FAIL (2025-08-07 23:36:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 428.594659ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..............................XXXXX..X......................] duration: 2.090603538s FAIL (2025-08-07 23:36:46 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.090649874s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................XXX.X.X......................] duration: 1.007517779s FAIL (2025-08-07 23:36:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== A5NM4A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.091791202 (6h51m59.678847453s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092306323 (6h51m59.679362574s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092699598 (6h51m59.679755839s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093097069 (6h51m59.68015331s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093493069 (6h51m59.68054931s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093847771 (6h51m59.680904022s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094203605 (6h51m59.681259855s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094597400 (6h51m59.681653641s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094954427 (6h51m59.682010668s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095373879 (6h51m59.68243012s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095742858 (6h51m59.6827991s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096084166 (6h51m59.683140377s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096399684 (6h51m59.683455905s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096721615 (6h51m59.683777826s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097044478 (6h51m59.684100699s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097416412 (6h51m59.684472633s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.172312403 (6h51m59.759368613s) [upload=7040]: Next returned nil segment Aug 7 23:18:46.172323192 (6h51m59.759379403s) [upload=7040]: Waiting for error group managing segments... Aug 7 23:18:46.182690655 (6h51m59.769746865s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.188105241 (6h51m59.775161462s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.196654078 (6h51m59.783710289s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198492931 (6h51m59.785549152s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198779515 (6h51m59.785835726s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.204927762 (6h51m59.791983983s) [noise=true , address=104.36.82.52:4099 , piece_id=H2WVX26COCLZ7KH6AJLYBN4H6JRIUF5IXYED6DRSCINOVGA5NM4A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.208251176 (6h51m59.795307387s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225240327 (6h51m59.812296549s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225308985 (6h51m59.812365196s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.245212838 (6h51m59.832269059s) [noise=true , address=104.36.82.4:4818 , piece_id=U5SIEBGYEKIRZ4FD46ZT3L3AVH7MIVC3JRKPC7V46F6MGFT5QFXQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.262860569 (6h51m59.84991678s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.287460286 (6h51m59.874516498s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.323021195 (6h51m59.910077436s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337567646 (6h51m59.924623857s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337650712 (6h51m59.924706923s) [seg_pos={0 0}, upload=7040]: Segment reached optimal threshold of 14 pieces. Aug 7 23:18:46.339154849 (6h51m59.92621106s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Lp3H6533e4oUt4mbiWJMjLRi5fb2wQr7RrntSpRm9xqF1TcEi): protocol: rpc: dial tcp 104.36.82.61:4379: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339496567 (6h51m59.926552778s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kNK8y5et7MiiWNDT6iaC8YBAk3PX6WX1XSuU4QMSFBp3KymBb): protocol: read tcp 5.161.89.252:19812->64.49.13.154:4930: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339638873 (6h51m59.926695084s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rUdrYUJgCF8aa3MdNe82dT765nqW46by7mJ2phMSpfHAhzgU2): protocol: read tcp 5.161.89.252:12938->104.36.82.46:4512: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339695828 (6h51m59.92675204s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ckJ4ZkPjcb4ZiJ2EauhxsyBMNvGWaVeJxvmiFGYhbxiaLT5uy): protocol: read tcp 5.161.89.252:28596->64.49.13.155:4922: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339725374 (6h51m59.926781605s) [seg_pos={0 0}, upload=7040]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:18:46.339804101 (6h51m59.926860312s) [upload=7040]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:46.339836361 (6h51m59.926892572s) [upload=7040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:46.393522662 (6h51m59.980578873s) [upload=7040]: Done uploading segments... Aug 7 23:18:46.393540715 (6h51m59.980596927s) [upload=7040]: Upload finished. err: <nil> Aug 7 23:20:00.141045360 (6h53m13.728101581s) [upload=7062]: Starting upload Aug 7 23:20:00.141098730 (6h53m13.728154951s) [upload=7062]: Done starting upload Aug 7 23:20:00.141249572 (6h53m13.728305793s) [upload=7062]: Uploading segments... Aug 7 23:20:00.141576813 (6h53m13.728633034s) [upload=7062]: Got next segment. Inline: false Aug 7 23:20:00.141621376 (6h53m13.728677597s) [upload=7062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:00.141625283 (6h53m13.728681494s) [upload=7062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:21:00.122447463 (6h54m13.709503674s) [upload=7062]: Next segment err: context deadline exceeded Aug 7 23:21:00.122599688 (6h54m13.709655909s) [upload=7062]: Done uploading segments... Aug 7 23:21:00.122624775 (6h54m13.709680986s) [upload=7062]: Upload finished. err: context deadline exceeded Duration: 1.007550701s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................X....XXX.X.....................] duration: 732.487653ms FAIL (2025-08-07 23:36:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 732.506718ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [..........................XXXXXX.X..........................] duration: 177.494459ms FAIL (2025-08-07 23:36:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 177.5215ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [...........................XXXX.X..X..X.....................] duration: 228.631381ms FAIL (2025-08-07 23:36:32 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 228.654854ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.054546135s FAIL (2025-08-07 23:36:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.054564938s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........................XXXXXX.X.X........................] duration: 1.86022339s FAIL (2025-08-07 23:36:29 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.860258566s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [..........................XXXXX.XX..........................] duration: 368.940118ms FAIL (2025-08-07 23:36:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.629132566 (6h43m50.216188797s) [upload=6906]: Upload finished. err: <nil> Aug 7 23:11:05.687805040 (6h44m19.274861281s) [upload=6916]: Starting upload Aug 7 23:11:05.687992249 (6h44m19.27504857s) [upload=6916]: Done starting upload Aug 7 23:11:05.688099038 (6h44m19.275155279s) [upload=6916]: Uploading segments... Aug 7 23:11:05.688157528 (6h44m19.275213799s) [upload=6916]: Got next segment. Inline: true Aug 7 23:11:05.688166765 (6h44m19.275223017s) [upload=6916]: Waiting for error group managing segments... Aug 7 23:11:05.688260139 (6h44m19.27531638s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:05.688269687 (6h44m19.275325969s) [upload=6916]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:05.688275919 (6h44m19.27533216s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:05.740655239 (6h44m19.327711451s) [upload=6916]: Done uploading segments... Aug 7 23:11:05.740669697 (6h44m19.327725918s) [upload=6916]: Upload finished. err: <nil> Aug 7 23:12:08.089702100 (6h45m21.67675834s) [upload=6934]: Starting upload Aug 7 23:12:08.089804582 (6h45m21.676860822s) [upload=6934]: Done starting upload Aug 7 23:12:08.089877487 (6h45m21.676933729s) [upload=6934]: Uploading segments... Aug 7 23:12:08.089928542 (6h45m21.676984783s) [upload=6934]: Got next segment. Inline: true Aug 7 23:12:08.089934242 (6h45m21.676990484s) [upload=6934]: Waiting for error group managing segments... Aug 7 23:12:08.089980470 (6h45m21.67703671s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:08.089986311 (6h45m21.677042551s) [upload=6934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:08.089990338 (6h45m21.677046579s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:08.195718921 (6h45m21.782775162s) [upload=6934]: Done uploading segments... Aug 7 23:12:08.195734549 (6h45m21.78279079s) [upload=6934]: Upload finished. err: <nil> Aug 7 23:12:48.910499617 (6h46m2.497555858s) [upload=6943]: Starting upload Aug 7 23:12:48.910615834 (6h46m2.497672085s) [upload=6943]: Done starting upload Aug 7 23:12:48.910681648 (6h46m2.497737898s) [upload=6943]: Uploading segments... Aug 7 23:12:48.910722363 (6h46m2.497778604s) [upload=6943]: Got next segment. Inline: true Aug 7 23:12:48.910727974 (6h46m2.497784214s) [upload=6943]: Waiting for error group managing segments... Aug 7 23:12:48.910775202 (6h46m2.497831443s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:48.910780953 (6h46m2.497837204s) [upload=6943]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:48.910784499 (6h46m2.497840741s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.893357638 (6h46m32.480413889s) [upload=6943]: Done uploading segments... Aug 7 23:13:18.893388065 (6h46m32.480444306s) [upload=6943]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:18.972282939 (6h46m32.559339181s) [upload=6953]: Starting upload Aug 7 23:13:18.972362728 (6h46m32.55941897s) [upload=6953]: Done starting upload Aug 7 23:13:18.972440393 (6h46m32.559496634s) [upload=6953]: Uploading segments... Aug 7 23:13:18.972481941 (6h46m32.559538181s) [upload=6953]: Got next segment. Inline: true Aug 7 23:13:18.972487681 (6h46m32.559543923s) [upload=6953]: Waiting for error group managing segments... Aug 7 23:13:18.972531043 (6h46m32.559587284s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.972536602 (6h46m32.559592843s) [upload=6953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.972540159 (6h46m32.5595964s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:48.959236568 (6h47m2.546292809s) [upload=6953]: Done uploading segments... Aug 7 23:13:48.959259700 (6h47m2.546315941s) [upload=6953]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:04.749315126 (6h47m18.336371367s) [upload=6964]: Starting upload Aug 7 23:14:04.749394674 (6h47m18.336450915s) [upload=6964]: Done starting upload Aug 7 23:14:04.749474242 (6h47m18.336530484s) [upload=6964]: Uploading segments... Aug 7 23:14:04.749532461 (6h47m18.336588702s) [upload=6964]: Got next segment. Inline: true Aug 7 23:14:04.749538312 (6h47m18.336594553s) [upload=6964]: Waiting for error group managing segments... Aug 7 23:14:04.749584738 (6h47m18.33664098s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:04.749590058 (6h47m18.3366463s) [upload=6964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:04.749593755 (6h47m18.336649997s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:34.737068324 (6h47m48.324124536s) [upload=6964]: Done uploading segments... Aug 7 23:14:34.737089104 (6h47m48.324145314s) [upload=6964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:37.277141003 (6h47m50.864197244s) [upload=6974]: Starting upload Aug 7 23:14:37.277320197 (6h47m50.864376439s) [upload=6974]: Done starting upload Aug 7 23:14:37.277421095 (6h47m50.864477337s) [upload=6974]: Uploading segments... Aug 7 23:14:37.277519849 (6h47m50.864576091s) [upload=6974]: Got next segment. Inline: true Aug 7 23:14:37.277527635 (6h47m50.864583875s) [upload=6974]: Waiting for error group managing segments... Aug 7 23:14:37.277578299 (6h47m50.86463454s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:37.277584181 (6h47m50.864640421s) [upload=6974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:37.277587807 (6h47m50.864644049s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:07.263997045 (6h48m20.851053287s) [upload=6974]: Done uploading segments... Aug 7 23:15:07.264036289 (6h48m20.85109253s) [upload=6974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:40.767391975 (6h48m54.354448226s) [upload=6993]: Starting upload Aug 7 23:15:40.767481722 (6h48m54.354537963s) [upload=6993]: Done starting upload Aug 7 23:15:40.767544320 (6h48m54.35460057s) [upload=6993]: Uploading segments... Aug 7 23:15:40.767585375 (6h48m54.354641617s) [upload=6993]: Got next segment. Inline: true Aug 7 23:15:40.767591397 (6h48m54.354647638s) [upload=6993]: Waiting for error group managing segments... Aug 7 23:15:40.767637002 (6h48m54.354693244s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:40.767642423 (6h48m54.354698664s) [upload=6993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:40.767646530 (6h48m54.354702782s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:10.754586120 (6h49m24.341642341s) [upload=6993]: Done uploading segments... Aug 7 23:16:10.754601419 (6h49m24.34165763s) [upload=6993]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:26.719060301 (6h49m40.306116542s) [upload=7005]: Starting upload Aug 7 23:16:26.719141191 (6h49m40.306197433s) [upload=7005]: Done starting upload Aug 7 23:16:26.719210051 (6h49m40.306266292s) [upload=7005]: Uploading segments... Aug 7 23:16:26.719285351 (6h49m40.306341592s) [upload=7005]: Got next segment. Inline: true Aug 7 23:16:26.719292685 (6h49m40.306348926s) [upload=7005]: Waiting for error group managing segments... Aug 7 23:16:26.719340825 (6h49m40.306397066s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:26.719347277 (6h49m40.306403518s) [upload=7005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:26.719351064 (6h49m40.306407305s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:26.802515608 (6h49m40.389571849s) [upload=7005]: Done uploading segments... Aug 7 23:16:26.802535986 (6h49m40.389592227s) [upload=7005]: Upload finished. err: <nil> Aug 7 23:16:42.191377275 (6h49m55.778433516s) [upload=7011]: Starting upload Aug 7 23:16:42.191466421 (6h49m55.778522662s) [upload=7011]: Done starting upload Aug 7 23:16:42.191531582 (6h49m55.778587803s) [upload=7011]: Uploading segments... Aug 7 23:16:42.191607003 (6h49m55.778663224s) [upload=7011]: Got next segment. Inline: true Aug 7 23:16:42.191612414 (6h49m55.778668635s) [upload=7011]: Waiting for error group managing segments... Aug 7 23:16:42.191651346 (6h49m55.778707556s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:42.191656575 (6h49m55.778712787s) [upload=7011]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:42.191659481 (6h49m55.778715702s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:12.173345160 (6h50m25.760401401s) [upload=7011]: Done uploading segments... Aug 7 23:17:12.173364576 (6h50m25.760420816s) [upload=7011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:12.261509015 (6h50m25.848565257s) [upload=7016]: Starting upload Aug 7 23:17:12.261597881 (6h50m25.848654122s) [upload=7016]: Done starting upload Aug 7 23:17:12.261665168 (6h50m25.848721419s) [upload=7016]: Uploading segments... Aug 7 23:17:12.261708137 (6h50m25.848764388s) [upload=7016]: Got next segment. Inline: true Aug 7 23:17:12.261713487 (6h50m25.848769728s) [upload=7016]: Waiting for error group managing segments... Aug 7 23:17:12.261755746 (6h50m25.848811988s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:12.261761196 (6h50m25.848817448s) [upload=7016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:12.261764593 (6h50m25.848820834s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:42.241895867 (6h50m55.828952078s) [upload=7016]: Done uploading segments... Aug 7 23:17:42.241913931 (6h50m55.828970141s) [upload=7016]: Upload finished. err: metaclient: context deadline exceeded Duration: 368.960537ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................XXX.X.X.....................] duration: 1.326389513s FAIL (2025-08-07 23:36:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== A5NM4A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.091791202 (6h51m59.678847453s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092306323 (6h51m59.679362574s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092699598 (6h51m59.679755839s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093097069 (6h51m59.68015331s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093493069 (6h51m59.68054931s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093847771 (6h51m59.680904022s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094203605 (6h51m59.681259855s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094597400 (6h51m59.681653641s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094954427 (6h51m59.682010668s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095373879 (6h51m59.68243012s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095742858 (6h51m59.6827991s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096084166 (6h51m59.683140377s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096399684 (6h51m59.683455905s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096721615 (6h51m59.683777826s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097044478 (6h51m59.684100699s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097416412 (6h51m59.684472633s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.172312403 (6h51m59.759368613s) [upload=7040]: Next returned nil segment Aug 7 23:18:46.172323192 (6h51m59.759379403s) [upload=7040]: Waiting for error group managing segments... Aug 7 23:18:46.182690655 (6h51m59.769746865s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.188105241 (6h51m59.775161462s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.196654078 (6h51m59.783710289s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198492931 (6h51m59.785549152s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198779515 (6h51m59.785835726s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.204927762 (6h51m59.791983983s) [noise=true , address=104.36.82.52:4099 , piece_id=H2WVX26COCLZ7KH6AJLYBN4H6JRIUF5IXYED6DRSCINOVGA5NM4A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.208251176 (6h51m59.795307387s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225240327 (6h51m59.812296549s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225308985 (6h51m59.812365196s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.245212838 (6h51m59.832269059s) [noise=true , address=104.36.82.4:4818 , piece_id=U5SIEBGYEKIRZ4FD46ZT3L3AVH7MIVC3JRKPC7V46F6MGFT5QFXQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.262860569 (6h51m59.84991678s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.287460286 (6h51m59.874516498s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.323021195 (6h51m59.910077436s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337567646 (6h51m59.924623857s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337650712 (6h51m59.924706923s) [seg_pos={0 0}, upload=7040]: Segment reached optimal threshold of 14 pieces. Aug 7 23:18:46.339154849 (6h51m59.92621106s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Lp3H6533e4oUt4mbiWJMjLRi5fb2wQr7RrntSpRm9xqF1TcEi): protocol: rpc: dial tcp 104.36.82.61:4379: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339496567 (6h51m59.926552778s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kNK8y5et7MiiWNDT6iaC8YBAk3PX6WX1XSuU4QMSFBp3KymBb): protocol: read tcp 5.161.89.252:19812->64.49.13.154:4930: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339638873 (6h51m59.926695084s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rUdrYUJgCF8aa3MdNe82dT765nqW46by7mJ2phMSpfHAhzgU2): protocol: read tcp 5.161.89.252:12938->104.36.82.46:4512: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339695828 (6h51m59.92675204s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ckJ4ZkPjcb4ZiJ2EauhxsyBMNvGWaVeJxvmiFGYhbxiaLT5uy): protocol: read tcp 5.161.89.252:28596->64.49.13.155:4922: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339725374 (6h51m59.926781605s) [seg_pos={0 0}, upload=7040]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:18:46.339804101 (6h51m59.926860312s) [upload=7040]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:46.339836361 (6h51m59.926892572s) [upload=7040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:46.393522662 (6h51m59.980578873s) [upload=7040]: Done uploading segments... Aug 7 23:18:46.393540715 (6h51m59.980596927s) [upload=7040]: Upload finished. err: <nil> Aug 7 23:20:00.141045360 (6h53m13.728101581s) [upload=7062]: Starting upload Aug 7 23:20:00.141098730 (6h53m13.728154951s) [upload=7062]: Done starting upload Aug 7 23:20:00.141249572 (6h53m13.728305793s) [upload=7062]: Uploading segments... Aug 7 23:20:00.141576813 (6h53m13.728633034s) [upload=7062]: Got next segment. Inline: false Aug 7 23:20:00.141621376 (6h53m13.728677597s) [upload=7062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:00.141625283 (6h53m13.728681494s) [upload=7062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:21:00.122447463 (6h54m13.709503674s) [upload=7062]: Next segment err: context deadline exceeded Aug 7 23:21:00.122599688 (6h54m13.709655909s) [upload=7062]: Done uploading segments... Aug 7 23:21:00.122624775 (6h54m13.709680986s) [upload=7062]: Upload finished. err: context deadline exceeded Duration: 1.326417155s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 175.084351ms FAIL (2025-08-07 23:36:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 175.112594ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........................XXXXXX.X.X.......................] duration: 1.770548546s FAIL (2025-08-07 23:36:11 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.770573223s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [..............................X.XXXXX..X....................] duration: 739.912534ms FAIL (2025-08-07 23:36:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 739.926531ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................X.XX.X....X.X....................] duration: 464.982041ms FAIL (2025-08-07 23:36:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 464.998372ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................X....XXX.X....................] duration: 542.413642ms FAIL (2025-08-07 23:36:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 542.442816ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.174167627s FAIL (2025-08-07 23:35:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.174195459s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 144.677694ms FAIL (2025-08-07 23:35:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 144.714131ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................XXX.X.X....................] duration: 822.468209ms FAIL (2025-08-07 23:35:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== A5NM4A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.091791202 (6h51m59.678847453s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092306323 (6h51m59.679362574s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092699598 (6h51m59.679755839s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093097069 (6h51m59.68015331s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093493069 (6h51m59.68054931s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093847771 (6h51m59.680904022s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094203605 (6h51m59.681259855s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094597400 (6h51m59.681653641s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094954427 (6h51m59.682010668s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095373879 (6h51m59.68243012s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095742858 (6h51m59.6827991s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096084166 (6h51m59.683140377s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096399684 (6h51m59.683455905s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096721615 (6h51m59.683777826s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097044478 (6h51m59.684100699s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097416412 (6h51m59.684472633s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.172312403 (6h51m59.759368613s) [upload=7040]: Next returned nil segment Aug 7 23:18:46.172323192 (6h51m59.759379403s) [upload=7040]: Waiting for error group managing segments... Aug 7 23:18:46.182690655 (6h51m59.769746865s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.188105241 (6h51m59.775161462s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.196654078 (6h51m59.783710289s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198492931 (6h51m59.785549152s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198779515 (6h51m59.785835726s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.204927762 (6h51m59.791983983s) [noise=true , address=104.36.82.52:4099 , piece_id=H2WVX26COCLZ7KH6AJLYBN4H6JRIUF5IXYED6DRSCINOVGA5NM4A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.208251176 (6h51m59.795307387s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225240327 (6h51m59.812296549s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225308985 (6h51m59.812365196s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.245212838 (6h51m59.832269059s) [noise=true , address=104.36.82.4:4818 , piece_id=U5SIEBGYEKIRZ4FD46ZT3L3AVH7MIVC3JRKPC7V46F6MGFT5QFXQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.262860569 (6h51m59.84991678s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.287460286 (6h51m59.874516498s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.323021195 (6h51m59.910077436s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337567646 (6h51m59.924623857s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337650712 (6h51m59.924706923s) [seg_pos={0 0}, upload=7040]: Segment reached optimal threshold of 14 pieces. Aug 7 23:18:46.339154849 (6h51m59.92621106s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Lp3H6533e4oUt4mbiWJMjLRi5fb2wQr7RrntSpRm9xqF1TcEi): protocol: rpc: dial tcp 104.36.82.61:4379: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339496567 (6h51m59.926552778s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kNK8y5et7MiiWNDT6iaC8YBAk3PX6WX1XSuU4QMSFBp3KymBb): protocol: read tcp 5.161.89.252:19812->64.49.13.154:4930: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339638873 (6h51m59.926695084s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rUdrYUJgCF8aa3MdNe82dT765nqW46by7mJ2phMSpfHAhzgU2): protocol: read tcp 5.161.89.252:12938->104.36.82.46:4512: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339695828 (6h51m59.92675204s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ckJ4ZkPjcb4ZiJ2EauhxsyBMNvGWaVeJxvmiFGYhbxiaLT5uy): protocol: read tcp 5.161.89.252:28596->64.49.13.155:4922: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339725374 (6h51m59.926781605s) [seg_pos={0 0}, upload=7040]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:18:46.339804101 (6h51m59.926860312s) [upload=7040]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:46.339836361 (6h51m59.926892572s) [upload=7040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:46.393522662 (6h51m59.980578873s) [upload=7040]: Done uploading segments... Aug 7 23:18:46.393540715 (6h51m59.980596927s) [upload=7040]: Upload finished. err: <nil> Aug 7 23:20:00.141045360 (6h53m13.728101581s) [upload=7062]: Starting upload Aug 7 23:20:00.141098730 (6h53m13.728154951s) [upload=7062]: Done starting upload Aug 7 23:20:00.141249572 (6h53m13.728305793s) [upload=7062]: Uploading segments... Aug 7 23:20:00.141576813 (6h53m13.728633034s) [upload=7062]: Got next segment. Inline: false Aug 7 23:20:00.141621376 (6h53m13.728677597s) [upload=7062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:00.141625283 (6h53m13.728681494s) [upload=7062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:21:00.122447463 (6h54m13.709503674s) [upload=7062]: Next segment err: context deadline exceeded Aug 7 23:21:00.122599688 (6h54m13.709655909s) [upload=7062]: Done uploading segments... Aug 7 23:21:00.122624775 (6h54m13.709680986s) [upload=7062]: Upload finished. err: context deadline exceeded Duration: 822.489369ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [............................XXXX.X..X..X....................] duration: 190.676788ms FAIL (2025-08-07 23:35:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 190.702907ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................X.XX.X....X.X...................] duration: 876.569907ms FAIL (2025-08-07 23:35:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 876.591408ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...............................XXXXX..X.....................] duration: 1.930781652s FAIL (2025-08-07 23:35:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.930803443s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [...............................X.XXXXX..X...................] duration: 776.416231ms FAIL (2025-08-07 23:35:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 776.461555ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [...........................XXXXX.XX.........................] duration: 161.812807ms FAIL (2025-08-07 23:35:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.629132566 (6h43m50.216188797s) [upload=6906]: Upload finished. err: <nil> Aug 7 23:11:05.687805040 (6h44m19.274861281s) [upload=6916]: Starting upload Aug 7 23:11:05.687992249 (6h44m19.27504857s) [upload=6916]: Done starting upload Aug 7 23:11:05.688099038 (6h44m19.275155279s) [upload=6916]: Uploading segments... Aug 7 23:11:05.688157528 (6h44m19.275213799s) [upload=6916]: Got next segment. Inline: true Aug 7 23:11:05.688166765 (6h44m19.275223017s) [upload=6916]: Waiting for error group managing segments... Aug 7 23:11:05.688260139 (6h44m19.27531638s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:05.688269687 (6h44m19.275325969s) [upload=6916]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:05.688275919 (6h44m19.27533216s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:05.740655239 (6h44m19.327711451s) [upload=6916]: Done uploading segments... Aug 7 23:11:05.740669697 (6h44m19.327725918s) [upload=6916]: Upload finished. err: <nil> Aug 7 23:12:08.089702100 (6h45m21.67675834s) [upload=6934]: Starting upload Aug 7 23:12:08.089804582 (6h45m21.676860822s) [upload=6934]: Done starting upload Aug 7 23:12:08.089877487 (6h45m21.676933729s) [upload=6934]: Uploading segments... Aug 7 23:12:08.089928542 (6h45m21.676984783s) [upload=6934]: Got next segment. Inline: true Aug 7 23:12:08.089934242 (6h45m21.676990484s) [upload=6934]: Waiting for error group managing segments... Aug 7 23:12:08.089980470 (6h45m21.67703671s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:08.089986311 (6h45m21.677042551s) [upload=6934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:08.089990338 (6h45m21.677046579s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:08.195718921 (6h45m21.782775162s) [upload=6934]: Done uploading segments... Aug 7 23:12:08.195734549 (6h45m21.78279079s) [upload=6934]: Upload finished. err: <nil> Aug 7 23:12:48.910499617 (6h46m2.497555858s) [upload=6943]: Starting upload Aug 7 23:12:48.910615834 (6h46m2.497672085s) [upload=6943]: Done starting upload Aug 7 23:12:48.910681648 (6h46m2.497737898s) [upload=6943]: Uploading segments... Aug 7 23:12:48.910722363 (6h46m2.497778604s) [upload=6943]: Got next segment. Inline: true Aug 7 23:12:48.910727974 (6h46m2.497784214s) [upload=6943]: Waiting for error group managing segments... Aug 7 23:12:48.910775202 (6h46m2.497831443s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:48.910780953 (6h46m2.497837204s) [upload=6943]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:48.910784499 (6h46m2.497840741s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.893357638 (6h46m32.480413889s) [upload=6943]: Done uploading segments... Aug 7 23:13:18.893388065 (6h46m32.480444306s) [upload=6943]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:18.972282939 (6h46m32.559339181s) [upload=6953]: Starting upload Aug 7 23:13:18.972362728 (6h46m32.55941897s) [upload=6953]: Done starting upload Aug 7 23:13:18.972440393 (6h46m32.559496634s) [upload=6953]: Uploading segments... Aug 7 23:13:18.972481941 (6h46m32.559538181s) [upload=6953]: Got next segment. Inline: true Aug 7 23:13:18.972487681 (6h46m32.559543923s) [upload=6953]: Waiting for error group managing segments... Aug 7 23:13:18.972531043 (6h46m32.559587284s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.972536602 (6h46m32.559592843s) [upload=6953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.972540159 (6h46m32.5595964s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:48.959236568 (6h47m2.546292809s) [upload=6953]: Done uploading segments... Aug 7 23:13:48.959259700 (6h47m2.546315941s) [upload=6953]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:04.749315126 (6h47m18.336371367s) [upload=6964]: Starting upload Aug 7 23:14:04.749394674 (6h47m18.336450915s) [upload=6964]: Done starting upload Aug 7 23:14:04.749474242 (6h47m18.336530484s) [upload=6964]: Uploading segments... Aug 7 23:14:04.749532461 (6h47m18.336588702s) [upload=6964]: Got next segment. Inline: true Aug 7 23:14:04.749538312 (6h47m18.336594553s) [upload=6964]: Waiting for error group managing segments... Aug 7 23:14:04.749584738 (6h47m18.33664098s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:04.749590058 (6h47m18.3366463s) [upload=6964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:04.749593755 (6h47m18.336649997s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:34.737068324 (6h47m48.324124536s) [upload=6964]: Done uploading segments... Aug 7 23:14:34.737089104 (6h47m48.324145314s) [upload=6964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:37.277141003 (6h47m50.864197244s) [upload=6974]: Starting upload Aug 7 23:14:37.277320197 (6h47m50.864376439s) [upload=6974]: Done starting upload Aug 7 23:14:37.277421095 (6h47m50.864477337s) [upload=6974]: Uploading segments... Aug 7 23:14:37.277519849 (6h47m50.864576091s) [upload=6974]: Got next segment. Inline: true Aug 7 23:14:37.277527635 (6h47m50.864583875s) [upload=6974]: Waiting for error group managing segments... Aug 7 23:14:37.277578299 (6h47m50.86463454s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:37.277584181 (6h47m50.864640421s) [upload=6974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:37.277587807 (6h47m50.864644049s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:07.263997045 (6h48m20.851053287s) [upload=6974]: Done uploading segments... Aug 7 23:15:07.264036289 (6h48m20.85109253s) [upload=6974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:40.767391975 (6h48m54.354448226s) [upload=6993]: Starting upload Aug 7 23:15:40.767481722 (6h48m54.354537963s) [upload=6993]: Done starting upload Aug 7 23:15:40.767544320 (6h48m54.35460057s) [upload=6993]: Uploading segments... Aug 7 23:15:40.767585375 (6h48m54.354641617s) [upload=6993]: Got next segment. Inline: true Aug 7 23:15:40.767591397 (6h48m54.354647638s) [upload=6993]: Waiting for error group managing segments... Aug 7 23:15:40.767637002 (6h48m54.354693244s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:40.767642423 (6h48m54.354698664s) [upload=6993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:40.767646530 (6h48m54.354702782s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:10.754586120 (6h49m24.341642341s) [upload=6993]: Done uploading segments... Aug 7 23:16:10.754601419 (6h49m24.34165763s) [upload=6993]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:26.719060301 (6h49m40.306116542s) [upload=7005]: Starting upload Aug 7 23:16:26.719141191 (6h49m40.306197433s) [upload=7005]: Done starting upload Aug 7 23:16:26.719210051 (6h49m40.306266292s) [upload=7005]: Uploading segments... Aug 7 23:16:26.719285351 (6h49m40.306341592s) [upload=7005]: Got next segment. Inline: true Aug 7 23:16:26.719292685 (6h49m40.306348926s) [upload=7005]: Waiting for error group managing segments... Aug 7 23:16:26.719340825 (6h49m40.306397066s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:26.719347277 (6h49m40.306403518s) [upload=7005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:26.719351064 (6h49m40.306407305s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:26.802515608 (6h49m40.389571849s) [upload=7005]: Done uploading segments... Aug 7 23:16:26.802535986 (6h49m40.389592227s) [upload=7005]: Upload finished. err: <nil> Aug 7 23:16:42.191377275 (6h49m55.778433516s) [upload=7011]: Starting upload Aug 7 23:16:42.191466421 (6h49m55.778522662s) [upload=7011]: Done starting upload Aug 7 23:16:42.191531582 (6h49m55.778587803s) [upload=7011]: Uploading segments... Aug 7 23:16:42.191607003 (6h49m55.778663224s) [upload=7011]: Got next segment. Inline: true Aug 7 23:16:42.191612414 (6h49m55.778668635s) [upload=7011]: Waiting for error group managing segments... Aug 7 23:16:42.191651346 (6h49m55.778707556s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:42.191656575 (6h49m55.778712787s) [upload=7011]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:42.191659481 (6h49m55.778715702s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:12.173345160 (6h50m25.760401401s) [upload=7011]: Done uploading segments... Aug 7 23:17:12.173364576 (6h50m25.760420816s) [upload=7011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:12.261509015 (6h50m25.848565257s) [upload=7016]: Starting upload Aug 7 23:17:12.261597881 (6h50m25.848654122s) [upload=7016]: Done starting upload Aug 7 23:17:12.261665168 (6h50m25.848721419s) [upload=7016]: Uploading segments... Aug 7 23:17:12.261708137 (6h50m25.848764388s) [upload=7016]: Got next segment. Inline: true Aug 7 23:17:12.261713487 (6h50m25.848769728s) [upload=7016]: Waiting for error group managing segments... Aug 7 23:17:12.261755746 (6h50m25.848811988s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:12.261761196 (6h50m25.848817448s) [upload=7016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:12.261764593 (6h50m25.848820834s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:42.241895867 (6h50m55.828952078s) [upload=7016]: Done uploading segments... Aug 7 23:17:42.241913931 (6h50m55.828970141s) [upload=7016]: Upload finished. err: metaclient: context deadline exceeded Duration: 161.843325ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................X....XXX.X...................] duration: 432.379414ms FAIL (2025-08-07 23:35:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 432.408488ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [................................XXXXX..X....................] duration: 1.695216919s FAIL (2025-08-07 23:35:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.695234833s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [...........................XXXXXX.X.........................] duration: 175.708775ms FAIL (2025-08-07 23:35:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 175.734725ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............................XXXXXX.X.X......................] duration: 1.806895881s FAIL (2025-08-07 23:35:09 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.806920968s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [............................XXXXXX.X........................] duration: 288.744577ms FAIL (2025-08-07 23:35:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 288.771147ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................X.XX.X....X.X..................] duration: 761.701621ms FAIL (2025-08-07 23:35:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 761.718912ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................X....XXX.X..................] duration: 397.288284ms FAIL (2025-08-07 23:34:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 397.312218ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.............................XXXX.X..X..X...................] duration: 188.772871ms FAIL (2025-08-07 23:34:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 188.792017ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 344.973705ms FAIL (2025-08-07 23:34:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 345.005623ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.............................XXXXXX.X.......................] duration: 188.356386ms FAIL (2025-08-07 23:34:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 188.382573ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................X....XXX.X.................] duration: 389.695971ms FAIL (2025-08-07 23:34:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 389.716869ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [............................XXXXX.XX........................] duration: 189.203575ms FAIL (2025-08-07 23:34:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.629132566 (6h43m50.216188797s) [upload=6906]: Upload finished. err: <nil> Aug 7 23:11:05.687805040 (6h44m19.274861281s) [upload=6916]: Starting upload Aug 7 23:11:05.687992249 (6h44m19.27504857s) [upload=6916]: Done starting upload Aug 7 23:11:05.688099038 (6h44m19.275155279s) [upload=6916]: Uploading segments... Aug 7 23:11:05.688157528 (6h44m19.275213799s) [upload=6916]: Got next segment. Inline: true Aug 7 23:11:05.688166765 (6h44m19.275223017s) [upload=6916]: Waiting for error group managing segments... Aug 7 23:11:05.688260139 (6h44m19.27531638s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:05.688269687 (6h44m19.275325969s) [upload=6916]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:05.688275919 (6h44m19.27533216s) [upload=6916]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:05.740655239 (6h44m19.327711451s) [upload=6916]: Done uploading segments... Aug 7 23:11:05.740669697 (6h44m19.327725918s) [upload=6916]: Upload finished. err: <nil> Aug 7 23:12:08.089702100 (6h45m21.67675834s) [upload=6934]: Starting upload Aug 7 23:12:08.089804582 (6h45m21.676860822s) [upload=6934]: Done starting upload Aug 7 23:12:08.089877487 (6h45m21.676933729s) [upload=6934]: Uploading segments... Aug 7 23:12:08.089928542 (6h45m21.676984783s) [upload=6934]: Got next segment. Inline: true Aug 7 23:12:08.089934242 (6h45m21.676990484s) [upload=6934]: Waiting for error group managing segments... Aug 7 23:12:08.089980470 (6h45m21.67703671s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:08.089986311 (6h45m21.677042551s) [upload=6934]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:08.089990338 (6h45m21.677046579s) [upload=6934]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:08.195718921 (6h45m21.782775162s) [upload=6934]: Done uploading segments... Aug 7 23:12:08.195734549 (6h45m21.78279079s) [upload=6934]: Upload finished. err: <nil> Aug 7 23:12:48.910499617 (6h46m2.497555858s) [upload=6943]: Starting upload Aug 7 23:12:48.910615834 (6h46m2.497672085s) [upload=6943]: Done starting upload Aug 7 23:12:48.910681648 (6h46m2.497737898s) [upload=6943]: Uploading segments... Aug 7 23:12:48.910722363 (6h46m2.497778604s) [upload=6943]: Got next segment. Inline: true Aug 7 23:12:48.910727974 (6h46m2.497784214s) [upload=6943]: Waiting for error group managing segments... Aug 7 23:12:48.910775202 (6h46m2.497831443s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:48.910780953 (6h46m2.497837204s) [upload=6943]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:48.910784499 (6h46m2.497840741s) [upload=6943]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:18.893357638 (6h46m32.480413889s) [upload=6943]: Done uploading segments... Aug 7 23:13:18.893388065 (6h46m32.480444306s) [upload=6943]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:18.972282939 (6h46m32.559339181s) [upload=6953]: Starting upload Aug 7 23:13:18.972362728 (6h46m32.55941897s) [upload=6953]: Done starting upload Aug 7 23:13:18.972440393 (6h46m32.559496634s) [upload=6953]: Uploading segments... Aug 7 23:13:18.972481941 (6h46m32.559538181s) [upload=6953]: Got next segment. Inline: true Aug 7 23:13:18.972487681 (6h46m32.559543923s) [upload=6953]: Waiting for error group managing segments... Aug 7 23:13:18.972531043 (6h46m32.559587284s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:18.972536602 (6h46m32.559592843s) [upload=6953]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:18.972540159 (6h46m32.5595964s) [upload=6953]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:48.959236568 (6h47m2.546292809s) [upload=6953]: Done uploading segments... Aug 7 23:13:48.959259700 (6h47m2.546315941s) [upload=6953]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:04.749315126 (6h47m18.336371367s) [upload=6964]: Starting upload Aug 7 23:14:04.749394674 (6h47m18.336450915s) [upload=6964]: Done starting upload Aug 7 23:14:04.749474242 (6h47m18.336530484s) [upload=6964]: Uploading segments... Aug 7 23:14:04.749532461 (6h47m18.336588702s) [upload=6964]: Got next segment. Inline: true Aug 7 23:14:04.749538312 (6h47m18.336594553s) [upload=6964]: Waiting for error group managing segments... Aug 7 23:14:04.749584738 (6h47m18.33664098s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:04.749590058 (6h47m18.3366463s) [upload=6964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:04.749593755 (6h47m18.336649997s) [upload=6964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:34.737068324 (6h47m48.324124536s) [upload=6964]: Done uploading segments... Aug 7 23:14:34.737089104 (6h47m48.324145314s) [upload=6964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:37.277141003 (6h47m50.864197244s) [upload=6974]: Starting upload Aug 7 23:14:37.277320197 (6h47m50.864376439s) [upload=6974]: Done starting upload Aug 7 23:14:37.277421095 (6h47m50.864477337s) [upload=6974]: Uploading segments... Aug 7 23:14:37.277519849 (6h47m50.864576091s) [upload=6974]: Got next segment. Inline: true Aug 7 23:14:37.277527635 (6h47m50.864583875s) [upload=6974]: Waiting for error group managing segments... Aug 7 23:14:37.277578299 (6h47m50.86463454s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:37.277584181 (6h47m50.864640421s) [upload=6974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:37.277587807 (6h47m50.864644049s) [upload=6974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:07.263997045 (6h48m20.851053287s) [upload=6974]: Done uploading segments... Aug 7 23:15:07.264036289 (6h48m20.85109253s) [upload=6974]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:40.767391975 (6h48m54.354448226s) [upload=6993]: Starting upload Aug 7 23:15:40.767481722 (6h48m54.354537963s) [upload=6993]: Done starting upload Aug 7 23:15:40.767544320 (6h48m54.35460057s) [upload=6993]: Uploading segments... Aug 7 23:15:40.767585375 (6h48m54.354641617s) [upload=6993]: Got next segment. Inline: true Aug 7 23:15:40.767591397 (6h48m54.354647638s) [upload=6993]: Waiting for error group managing segments... Aug 7 23:15:40.767637002 (6h48m54.354693244s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:40.767642423 (6h48m54.354698664s) [upload=6993]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:40.767646530 (6h48m54.354702782s) [upload=6993]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:10.754586120 (6h49m24.341642341s) [upload=6993]: Done uploading segments... Aug 7 23:16:10.754601419 (6h49m24.34165763s) [upload=6993]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:26.719060301 (6h49m40.306116542s) [upload=7005]: Starting upload Aug 7 23:16:26.719141191 (6h49m40.306197433s) [upload=7005]: Done starting upload Aug 7 23:16:26.719210051 (6h49m40.306266292s) [upload=7005]: Uploading segments... Aug 7 23:16:26.719285351 (6h49m40.306341592s) [upload=7005]: Got next segment. Inline: true Aug 7 23:16:26.719292685 (6h49m40.306348926s) [upload=7005]: Waiting for error group managing segments... Aug 7 23:16:26.719340825 (6h49m40.306397066s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:26.719347277 (6h49m40.306403518s) [upload=7005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:26.719351064 (6h49m40.306407305s) [upload=7005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:26.802515608 (6h49m40.389571849s) [upload=7005]: Done uploading segments... Aug 7 23:16:26.802535986 (6h49m40.389592227s) [upload=7005]: Upload finished. err: <nil> Aug 7 23:16:42.191377275 (6h49m55.778433516s) [upload=7011]: Starting upload Aug 7 23:16:42.191466421 (6h49m55.778522662s) [upload=7011]: Done starting upload Aug 7 23:16:42.191531582 (6h49m55.778587803s) [upload=7011]: Uploading segments... Aug 7 23:16:42.191607003 (6h49m55.778663224s) [upload=7011]: Got next segment. Inline: true Aug 7 23:16:42.191612414 (6h49m55.778668635s) [upload=7011]: Waiting for error group managing segments... Aug 7 23:16:42.191651346 (6h49m55.778707556s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:42.191656575 (6h49m55.778712787s) [upload=7011]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:42.191659481 (6h49m55.778715702s) [upload=7011]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:12.173345160 (6h50m25.760401401s) [upload=7011]: Done uploading segments... Aug 7 23:17:12.173364576 (6h50m25.760420816s) [upload=7011]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:12.261509015 (6h50m25.848565257s) [upload=7016]: Starting upload Aug 7 23:17:12.261597881 (6h50m25.848654122s) [upload=7016]: Done starting upload Aug 7 23:17:12.261665168 (6h50m25.848721419s) [upload=7016]: Uploading segments... Aug 7 23:17:12.261708137 (6h50m25.848764388s) [upload=7016]: Got next segment. Inline: true Aug 7 23:17:12.261713487 (6h50m25.848769728s) [upload=7016]: Waiting for error group managing segments... Aug 7 23:17:12.261755746 (6h50m25.848811988s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:12.261761196 (6h50m25.848817448s) [upload=7016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:12.261764593 (6h50m25.848820834s) [upload=7016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:42.241895867 (6h50m55.828952078s) [upload=7016]: Done uploading segments... Aug 7 23:17:42.241913931 (6h50m55.828970141s) [upload=7016]: Upload finished. err: metaclient: context deadline exceeded Duration: 189.21618ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................XXX.X.X...................] duration: 623.727641ms FAIL (2025-08-07 23:34:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== A5NM4A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.091791202 (6h51m59.678847453s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092306323 (6h51m59.679362574s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.092699598 (6h51m59.679755839s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093097069 (6h51m59.68015331s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093493069 (6h51m59.68054931s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.093847771 (6h51m59.680904022s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094203605 (6h51m59.681259855s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094597400 (6h51m59.681653641s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.094954427 (6h51m59.682010668s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095373879 (6h51m59.68243012s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.095742858 (6h51m59.6827991s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096084166 (6h51m59.683140377s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096399684 (6h51m59.683455905s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.096721615 (6h51m59.683777826s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097044478 (6h51m59.684100699s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.097416412 (6h51m59.684472633s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Uploading piece... Aug 7 23:18:46.172312403 (6h51m59.759368613s) [upload=7040]: Next returned nil segment Aug 7 23:18:46.172323192 (6h51m59.759379403s) [upload=7040]: Waiting for error group managing segments... Aug 7 23:18:46.182690655 (6h51m59.769746865s) [noise=true , address=45.91.116.9:4290 , piece_id=S3USZPGNHC7SX3HHEM6LBII54CGKRCQGLMQNRE7ZQXM3ZRLMJMSQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.188105241 (6h51m59.775161462s) [noise=true , address=104.36.82.7:4935 , piece_id=LVI53GXPJGQCYU5QXT2MRTPZJOQETBU3RUGVV2DGSN26NP6I3XVQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.196654078 (6h51m59.783710289s) [noise=true , address=45.91.116.13:4171 , piece_id=UNI5ZZNWXKPIKQXVVWNE3S7FUGM2XWNO7PSJDRWGBSOERCY22OPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198492931 (6h51m59.785549152s) [noise=true , address=104.36.82.58:4299 , piece_id=25QYBX42LIC7HWREJDDQWECRZFNZV3PBM2JITVH6R52TGPDEMQDA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.198779515 (6h51m59.785835726s) [noise=true , address=104.36.82.59:4307 , piece_id=DGJPGA6ETYMP34JBQE6WXC32PIXFWLUUKZNNWB2NXBWS727MJRPQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.204927762 (6h51m59.791983983s) [noise=true , address=104.36.82.52:4099 , piece_id=H2WVX26COCLZ7KH6AJLYBN4H6JRIUF5IXYED6DRSCINOVGA5NM4A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.208251176 (6h51m59.795307387s) [noise=true , address=104.36.82.16:4315 , piece_id=YDQ3SD75HCNENS667QTX2U5WCHQUFEEOX3WW6CXNVHKPJ7ZAE2YA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225240327 (6h51m59.812296549s) [noise=true , address=104.36.82.62:4439 , piece_id=P66ZY2SI3GROYBUTKW7HEEUYT6VD4WWY53PQEL43J25DQWHHGK5Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.225308985 (6h51m59.812365196s) [noise=true , address=104.36.82.3:4794 , piece_id=2M55ORW4IODVHZGVFIAOG6SDOHBT7NWX3X74ANGSUHIKY7ZLAFJA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.245212838 (6h51m59.832269059s) [noise=true , address=104.36.82.4:4818 , piece_id=U5SIEBGYEKIRZ4FD46ZT3L3AVH7MIVC3JRKPC7V46F6MGFT5QFXQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.262860569 (6h51m59.84991678s) [noise=true , address=104.36.82.9:4031 , piece_id=TRXII4JD2BKKTYJ3EVICMM7WGPELQNRU7Z7I23KKY2NFT4KN3JNA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.287460286 (6h51m59.874516498s) [noise=true , address=104.36.82.9:4036 , piece_id=KSI3DLD6I3TZUKTH7KQTLLBEIMILZ2GUHVNP5JU6MWNVRIEARHGQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.323021195 (6h51m59.910077436s) [noise=true , address=104.36.82.42:4815 , piece_id=NEGWCQ6XEFYUU5XIXZRYB5WVDRVF6ARBF6LMGC65AQHNGGJRJIXA, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337567646 (6h51m59.924623857s) [noise=true , address=79.127.201.215:4350 , piece_id=WSQK2BLHIRJ4J6FH22IEV64A3VV7N7RFNR2WI2XJXWNMSW2H765Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: <nil> Aug 7 23:18:46.337650712 (6h51m59.924706923s) [seg_pos={0 0}, upload=7040]: Segment reached optimal threshold of 14 pieces. Aug 7 23:18:46.339154849 (6h51m59.92621106s) [noise=true , address=104.36.82.61:4379 , piece_id=W2XB5LCHE7BIGVF53UOKTBS2E2ZQJ2FZHLRDUKQHPUENZQELPU2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Lp3H6533e4oUt4mbiWJMjLRi5fb2wQr7RrntSpRm9xqF1TcEi): protocol: rpc: dial tcp 104.36.82.61:4379: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339496567 (6h51m59.926552778s) [noise=true , address=64.49.13.154:4930 , piece_id=4D74XBIPQHETEYR7WQSZHGFHW42EGNAVYDO5NSBKAFP44BK4JR2Q, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kNK8y5et7MiiWNDT6iaC8YBAk3PX6WX1XSuU4QMSFBp3KymBb): protocol: read tcp 5.161.89.252:19812->64.49.13.154:4930: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339638873 (6h51m59.926695084s) [noise=true , address=104.36.82.46:4512 , piece_id=X5SF67DUK4DXJICZCZVYBVTDFSS3HHFHXUEYJMMOMSVFORXUBS2A, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rUdrYUJgCF8aa3MdNe82dT765nqW46by7mJ2phMSpfHAhzgU2): protocol: read tcp 5.161.89.252:12938->104.36.82.46:4512: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339695828 (6h51m59.92675204s) [noise=true , address=64.49.13.155:4922 , piece_id=XWJAU4RZXJAW2IM5CUGFI3MAK5GYEM6AMWMO6AO7PK6MYA47ABQQ, seg_pos={0 0}, upload=7040]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ckJ4ZkPjcb4ZiJ2EauhxsyBMNvGWaVeJxvmiFGYhbxiaLT5uy): protocol: read tcp 5.161.89.252:28596->64.49.13.155:4922: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:18:46.339725374 (6h51m59.926781605s) [seg_pos={0 0}, upload=7040]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:18:46.339804101 (6h51m59.926860312s) [upload=7040]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:46.339836361 (6h51m59.926892572s) [upload=7040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:46.393522662 (6h51m59.980578873s) [upload=7040]: Done uploading segments... Aug 7 23:18:46.393540715 (6h51m59.980596927s) [upload=7040]: Upload finished. err: <nil> Aug 7 23:20:00.141045360 (6h53m13.728101581s) [upload=7062]: Starting upload Aug 7 23:20:00.141098730 (6h53m13.728154951s) [upload=7062]: Done starting upload Aug 7 23:20:00.141249572 (6h53m13.728305793s) [upload=7062]: Uploading segments... Aug 7 23:20:00.141576813 (6h53m13.728633034s) [upload=7062]: Got next segment. Inline: false Aug 7 23:20:00.141621376 (6h53m13.728677597s) [upload=7062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:20:00.141625283 (6h53m13.728681494s) [upload=7062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:21:00.122447463 (6h54m13.709503674s) [upload=7062]: Next segment err: context deadline exceeded Aug 7 23:21:00.122599688 (6h54m13.709655909s) [upload=7062]: Done uploading segments... Aug 7 23:21:00.122624775 (6h54m13.709680986s) [upload=7062]: Upload finished. err: context deadline exceeded Duration: 623.749041ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X......................................................X....] duration: 1.129514175s FAIL (2025-08-07 23:34:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.129545655s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)