Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.090907317s FAIL (2025-08-07 23:55:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.090935279s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...XXXX.X..X..X......................................XXX.X.X] duration: 30.001332937s FAIL (2025-08-07 23:55:37 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001353746s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [....X.XXXXX..X..............................................] duration: 372.416746ms FAIL (2025-08-07 23:55:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 372.711828ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [........XXX.X.X.........................................XX..] duration: 1.365937922s FAIL (2025-08-07 23:55:29 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ading piece... Aug 7 23:51:17.347399839 (7h24m30.934456051s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347417763 (7h24m30.934473974s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347625520 (7h24m30.934681731s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347755754 (7h24m30.934811985s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347968761 (7h24m30.935024982s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348133560 (7h24m30.935189781s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348193100 (7h24m30.935249311s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348536501 (7h24m30.935592722s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348656385 (7h24m30.935712596s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348854374 (7h24m30.935910585s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348919236 (7h24m30.935975457s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.430583697 (7h24m31.017639918s) [upload=7600]: Next returned nil segment Aug 7 23:51:17.430596350 (7h24m31.017652571s) [upload=7600]: Waiting for error group managing segments... Aug 7 23:51:17.450496095 (7h24m31.037552306s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.457273115 (7h24m31.044329336s) [noise=true , address=45.91.116.11:4121 , piece_id=UI4HOUG4KLWC3LFGDVZ3C3IUMB2DGJP2QOTIOLWS5C2HUAMSATMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.458658320 (7h24m31.045714532s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.464324767 (7h24m31.051380978s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.469258046 (7h24m31.056314256s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.470547041 (7h24m31.057603252s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.482895480 (7h24m31.069951691s) [noise=true , address=104.36.82.59:4327 , piece_id=5AM7TVDV5ORD5WPHKEHOM652JETCJ6VH3JOJ6HPJTIOTABTYVELA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.503848269 (7h24m31.090904481s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.589438619 (7h24m31.17649484s) [noise=true , address=79.127.201.215:4343 , piece_id=6IVWBNSJNI4ZAEIWFTRYUHIHZ35YTTZF3FTEF7G76YEBUP2NBJFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.633863581 (7h24m31.220919802s) [noise=true , address=104.36.82.42:4801 , piece_id=ILNDV4CZCPEST6R4DUTSHQ5OJS2N4BLPRLDYIYNBTZKRR4RSCOFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.636493760 (7h24m31.22354997s) [noise=true , address=104.36.82.44:4904 , piece_id=TFXFZ53DGPOVVXTTZMRMTJR53DGXNHWUF43I7OCC4YNCRW4QUCKA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.637613389 (7h24m31.224669611s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.654845844 (7h24m31.241902065s) [noise=true , address=104.36.82.22:4524 , piece_id=66YIUJKSDV52T5NMMFHYCZH7L34UKH3RLIS5WZ3VWRBL3KEE7UWQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853873712 (7h24m31.440929933s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853888850 (7h24m31.440945061s) [seg_pos={0 0}, upload=7600]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:17.853987013 (7h24m31.441043234s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:17.854295209 (7h24m31.44135142s) [noise=true , address=104.36.82.46:4499 , piece_id=6OBTEQJKT4AJRGUT27EYWNLG4E2TKBCQBRGXTTEOU4SJY2SWUTFQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection Aug 7 23:51:17.854338279 (7h24m31.44139449s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zkqEMSKx7zSHPFNZ51b5TLzL2cFQL8VyxAG14Lg1pT8zAu3tc): protocol: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection Aug 7 23:51:17.854399554 (7h24m31.441455765s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gPjPGuueZnizLT2QvJN9y7riwKiW5TWMkQtmGGPP7VZvkNoB7): protocol: read tcp 5.161.89.252:35428->104.36.82.15:4288: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:51:17.854419941 (7h24m31.441476162s) [seg_pos={0 0}, upload=7600]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:17.854472530 (7h24m31.441528751s) [upload=7600]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:17.854476738 (7h24m31.441532959s) [upload=7600]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:17.900817215 (7h24m31.487873465s) [upload=7600]: Done uploading segments... Aug 7 23:51:17.900834827 (7h24m31.487891079s) [upload=7600]: Upload finished. err: <nil> Aug 7 23:52:11.382772324 (7h25m24.969828534s) [upload=7616]: Starting upload Aug 7 23:52:11.382828459 (7h25m24.96988468s) [upload=7616]: Done starting upload Aug 7 23:52:11.382871529 (7h25m24.96992774s) [upload=7616]: Uploading segments... Aug 7 23:52:11.383053639 (7h25m24.97010986s) [upload=7616]: Got next segment. Inline: false Aug 7 23:52:11.383080680 (7h25m24.970136891s) [upload=7616]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:11.383084406 (7h25m24.970140617s) [upload=7616]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:11.371970391 (7h26m24.959026631s) [upload=7616]: Next segment err: context deadline exceeded Aug 7 23:53:11.372056972 (7h26m24.959113223s) [upload=7616]: Done uploading segments... Aug 7 23:53:11.372090856 (7h26m24.959147096s) [upload=7616]: Upload finished. err: context deadline exceeded Aug 7 23:53:11.462534524 (7h26m25.049590775s) [upload=7633]: Starting upload Aug 7 23:53:11.462662182 (7h26m25.049718423s) [upload=7633]: Done starting upload Aug 7 23:53:11.462730148 (7h26m25.0497864s) [upload=7633]: Uploading segments... Aug 7 23:53:11.463218160 (7h26m25.050274451s) [upload=7633]: Got next segment. Inline: false Aug 7 23:53:11.463267292 (7h26m25.050323533s) [upload=7633]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.463276689 (7h26m25.050332931s) [upload=7633]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:11.448954917 (7h27m25.036011128s) [upload=7633]: Next segment err: context deadline exceeded Aug 7 23:54:11.449030117 (7h27m25.036086328s) [upload=7633]: Done uploading segments... Aug 7 23:54:11.449047229 (7h27m25.03610344s) [upload=7633]: Upload finished. err: context deadline exceeded Duration: 1.365959312s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [..XXXXXX.X...............................................XX.] duration: 228.131792ms FAIL (2025-08-07 23:55:12 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 95s) [upload=7512]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:05.713026270 (7h19m19.300082522s) [upload=7512]: Done uploading segments... Aug 7 23:46:05.713042230 (7h19m19.300098472s) [upload=7512]: Upload finished. err: <nil> Aug 7 23:46:59.827069649 (7h20m13.41412587s) [upload=7526]: Starting upload Aug 7 23:46:59.827112839 (7h20m13.414169051s) [upload=7526]: Done starting upload Aug 7 23:46:59.827158284 (7h20m13.414214506s) [upload=7526]: Uploading segments... Aug 7 23:46:59.827184925 (7h20m13.414241135s) [upload=7526]: Got next segment. Inline: true Aug 7 23:46:59.827188771 (7h20m13.414244992s) [upload=7526]: Waiting for error group managing segments... Aug 7 23:46:59.827219097 (7h20m13.414275308s) [upload=7526]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:59.827222835 (7h20m13.414279056s) [upload=7526]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:59.827225119 (7h20m13.414281329s) [upload=7526]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:59.879507639 (7h20m13.46656385s) [upload=7526]: Done uploading segments... Aug 7 23:46:59.879523058 (7h20m13.466579269s) [upload=7526]: Upload finished. err: <nil> Aug 7 23:47:39.924312697 (7h20m53.511368928s) [upload=7536]: Starting upload Aug 7 23:47:39.924413405 (7h20m53.511469646s) [upload=7536]: Done starting upload Aug 7 23:47:39.924491079 (7h20m53.51154731s) [upload=7536]: Uploading segments... Aug 7 23:47:39.924700320 (7h20m53.511756551s) [upload=7536]: Got next segment. Inline: true Aug 7 23:47:39.924711842 (7h20m53.511768063s) [upload=7536]: Waiting for error group managing segments... Aug 7 23:47:39.924828248 (7h20m53.51188448s) [upload=7536]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:39.924836915 (7h20m53.511893156s) [upload=7536]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:39.924842024 (7h20m53.511898255s) [upload=7536]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Duration:* 228.147561ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [...XXXXXX.X...............................................XX] duration: 1m0.001690304s FAIL (2025-08-07 23:55:11 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 95s) [upload=7512]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:05.713026270 (7h19m19.300082522s) [upload=7512]: Done uploading segments... Aug 7 23:46:05.713042230 (7h19m19.300098472s) [upload=7512]: Upload finished. err: <nil> Aug 7 23:46:59.827069649 (7h20m13.41412587s) [upload=7526]: Starting upload Aug 7 23:46:59.827112839 (7h20m13.414169051s) [upload=7526]: Done starting upload Aug 7 23:46:59.827158284 (7h20m13.414214506s) [upload=7526]: Uploading segments... Aug 7 23:46:59.827184925 (7h20m13.414241135s) [upload=7526]: Got next segment. Inline: true Aug 7 23:46:59.827188771 (7h20m13.414244992s) [upload=7526]: Waiting for error group managing segments... Aug 7 23:46:59.827219097 (7h20m13.414275308s) [upload=7526]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:59.827222835 (7h20m13.414279056s) [upload=7526]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:59.827225119 (7h20m13.414281329s) [upload=7526]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:59.879507639 (7h20m13.46656385s) [upload=7526]: Done uploading segments... Aug 7 23:46:59.879523058 (7h20m13.466579269s) [upload=7526]: Upload finished. err: <nil> Aug 7 23:47:39.924312697 (7h20m53.511368928s) [upload=7536]: Starting upload Aug 7 23:47:39.924413405 (7h20m53.511469646s) [upload=7536]: Done starting upload Aug 7 23:47:39.924491079 (7h20m53.51154731s) [upload=7536]: Uploading segments... Aug 7 23:47:39.924700320 (7h20m53.511756551s) [upload=7536]: Got next segment. Inline: true Aug 7 23:47:39.924711842 (7h20m53.511768063s) [upload=7536]: Waiting for error group managing segments... Aug 7 23:47:39.924828248 (7h20m53.51188448s) [upload=7536]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:39.924836915 (7h20m53.511893156s) [upload=7536]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:39.924842024 (7h20m53.511898255s) [upload=7536]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.001703699s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X.XX.X....X.X..........................................X...] duration: 1.026320259s FAIL (2025-08-07 23:55:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.026338653s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXX.XX............................................XXXXXX] duration: 30.000769403s FAIL (2025-08-07 23:55:01 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 10 (7h20m13.662582431s) [upload=7527]: Done uploading segments... Aug 7 23:47:00.075540917 (7h20m13.662597138s) [upload=7527]: Upload finished. err: <nil> Aug 7 23:48:09.882765252 (7h21m23.469821463s) [upload=7545]: Starting upload Aug 7 23:48:09.882860198 (7h21m23.46991642s) [upload=7545]: Done starting upload Aug 7 23:48:09.882935729 (7h21m23.469991941s) [upload=7545]: Uploading segments... Aug 7 23:48:09.882965635 (7h21m23.470021856s) [upload=7545]: Got next segment. Inline: true Aug 7 23:48:09.882969723 (7h21m23.470025944s) [upload=7545]: Waiting for error group managing segments... Aug 7 23:48:09.883002865 (7h21m23.470059086s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:09.883006481 (7h21m23.470062702s) [upload=7545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:09.883009045 (7h21m23.470065266s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:09.951272229 (7h21m23.538328451s) [upload=7545]: Done uploading segments... Aug 7 23:48:09.951286926 (7h21m23.538343148s) [upload=7545]: Upload finished. err: <nil> Aug 7 23:48:58.772502770 (7h22m12.359559011s) [upload=7557]: Starting upload Aug 7 23:48:58.772574805 (7h22m12.359631046s) [upload=7557]: Done starting upload Aug 7 23:48:58.772645527 (7h22m12.359701767s) [upload=7557]: Uploading segments... Aug 7 23:48:58.772695300 (7h22m12.359751541s) [upload=7557]: Got next segment. Inline: true Aug 7 23:48:58.772701031 (7h22m12.359757271s) [upload=7557]: Waiting for error group managing segments... Aug 7 23:48:58.772770249 (7h22m12.359826501s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:58.772780228 (7h22m12.359836469s) [upload=7557]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:58.772787010 (7h22m12.359843252s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:58.840436652 (7h22m12.427492893s) [upload=7557]: Done uploading segments... Aug 7 23:48:58.840462661 (7h22m12.427518891s) [upload=7557]: Upload finished. err: <nil> Aug 7 23:50:01.971851413 (7h23m15.558907624s) [upload=7574]: Starting upload Aug 7 23:50:01.971890696 (7h23m15.558946907s) [upload=7574]: Done starting upload Aug 7 23:50:01.971977749 (7h23m15.559033961s) [upload=7574]: Uploading segments... Aug 7 23:50:01.972022943 (7h23m15.559079154s) [upload=7574]: Got next segment. Inline: true Aug 7 23:50:01.972026850 (7h23m15.559083062s) [upload=7574]: Waiting for error group managing segments... Aug 7 23:50:01.972060975 (7h23m15.559117185s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:01.972064511 (7h23m15.559120712s) [upload=7574]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:01.972066685 (7h23m15.559122896s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:02.063709030 (7h23m15.650765241s) [upload=7574]: Done uploading segments... Aug 7 23:50:02.063723547 (7h23m15.650779768s) [upload=7574]: Upload finished. err: <nil> Aug 7 23:50:41.160166556 (7h23m54.747222777s) [upload=7588]: Starting upload Aug 7 23:50:41.160220555 (7h23m54.747276777s) [upload=7588]: Done starting upload Aug 7 23:50:41.160264659 (7h23m54.74732088s) [upload=7588]: Uploading segments... Aug 7 23:50:41.160295956 (7h23m54.747352177s) [upload=7588]: Got next segment. Inline: true Aug 7 23:50:41.160299753 (7h23m54.747355964s) [upload=7588]: Waiting for error group managing segments... Aug 7 23:50:41.160331633 (7h23m54.747387844s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:41.160335861 (7h23m54.747392072s) [upload=7588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:41.160338396 (7h23m54.747394617s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:41.222859735 (7h23m54.809915976s) [upload=7588]: Done uploading segments... Aug 7 23:50:41.222879913 (7h23m54.809936154s) [upload=7588]: Upload finished. err: <nil> Aug 7 23:51:17.131690986 (7h24m30.718747197s) [upload=7599]: Starting upload Aug 7 23:51:17.131738926 (7h24m30.718795147s) [upload=7599]: Done starting upload Aug 7 23:51:17.131792947 (7h24m30.718849178s) [upload=7599]: Uploading segments... Aug 7 23:51:17.131835896 (7h24m30.718892117s) [upload=7599]: Got next segment. Inline: true Aug 7 23:51:17.131842027 (7h24m30.718898239s) [upload=7599]: Waiting for error group managing segments... Aug 7 23:51:17.131883385 (7h24m30.718939627s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:17.131889146 (7h24m30.718945388s) [upload=7599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:17.132006205 (7h24m30.719062435s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:38.787433326 (7h24m52.374489567s) [upload=7599]: Done uploading segments... Aug 7 23:51:38.787452212 (7h24m52.374508453s) [upload=7599]: Upload finished. err: <nil> Aug 7 23:52:00.256204647 (7h25m13.843260858s) [upload=7610]: Starting upload Aug 7 23:52:00.256263547 (7h25m13.843319758s) [upload=7610]: Done starting upload Aug 7 23:52:00.256316195 (7h25m13.843372406s) [upload=7610]: Uploading segments... Aug 7 23:52:00.256345671 (7h25m13.843401881s) [upload=7610]: Got next segment. Inline: true Aug 7 23:52:00.256349758 (7h25m13.843405969s) [upload=7610]: Waiting for error group managing segments... Aug 7 23:52:00.256399992 (7h25m13.843456203s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:00.256404780 (7h25m13.843460991s) [upload=7610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:00.256407806 (7h25m13.843464017s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:30.237600164 (7h25m43.824656416s) [upload=7610]: Done uploading segments... Aug 7 23:52:30.237630862 (7h25m43.824687113s) [upload=7610]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:53.146251246 (7h26m6.733307456s) [upload=7626]: Starting upload Aug 7 23:52:53.146307339 (7h26m6.733363561s) [upload=7626]: Done starting upload Aug 7 23:52:53.146350962 (7h26m6.733407173s) [upload=7626]: Uploading segments... Aug 7 23:52:53.146411344 (7h26m6.733467565s) [upload=7626]: Got next segment. Inline: true Aug 7 23:52:53.146415271 (7h26m6.733471493s) [upload=7626]: Waiting for error group managing segments... Aug 7 23:52:53.146449966 (7h26m6.733506177s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:53.146454174 (7h26m6.733510395s) [upload=7626]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:53.146456709 (7h26m6.73351293s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:23.132536521 (7h26m36.719592742s) [upload=7626]: Done uploading segments... Aug 7 23:53:23.132560907 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000779852s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [.....XXXXX..X............................................XXX] duration: 1m0.000920548s FAIL (2025-08-07 23:55:00 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000931317s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.....X.XXXXX..X.............................................] duration: 376.124477ms FAIL (2025-08-07 23:54:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 376.149834ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 133.66315ms FAIL (2025-08-07 23:54:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 133.723323ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..XXXXXX.X.X...........................................X.X..] duration: 1.740102805s FAIL (2025-08-07 23:54:48 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.740120539s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.041137846s FAIL (2025-08-07 23:54:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.041153565s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [....XXXX.X..X..X......................................XXX.X.] duration: 321.08864ms FAIL (2025-08-07 23:54:40 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 321.108097ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 11 out of 60 is failed (threshold=3) [.....XXXX.X..X..X......................................XXX.X] duration: 30.000572123s FAIL (2025-08-07 23:54:39 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000589657s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..X.XX.X....X.X..........................................X..] duration: 454.532543ms FAIL (2025-08-07 23:54:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.553602ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...XXXXX.XX............................................XXXXX] duration: 30.001415622s FAIL (2025-08-07 23:54:31 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ctCommit Aug 7 23:46:08.965075092 (7h19m22.552131313s) [upload=7514]: Done uploading segments... Aug 7 23:46:08.965095970 (7h19m22.552152191s) [upload=7514]: Upload finished. err: <nil> Aug 7 23:47:00.017908282 (7h20m13.604964493s) [upload=7527]: Starting upload Aug 7 23:47:00.017955210 (7h20m13.605011411s) [upload=7527]: Done starting upload Aug 7 23:47:00.017998241 (7h20m13.605054452s) [upload=7527]: Uploading segments... Aug 7 23:47:00.018026292 (7h20m13.605082503s) [upload=7527]: Got next segment. Inline: true Aug 7 23:47:00.018029438 (7h20m13.60508565s) [upload=7527]: Waiting for error group managing segments... Aug 7 23:47:00.018058453 (7h20m13.605114664s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:00.018062390 (7h20m13.605118601s) [upload=7527]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:00.018064304 (7h20m13.605120515s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:00.075526210 (7h20m13.662582431s) [upload=7527]: Done uploading segments... Aug 7 23:47:00.075540917 (7h20m13.662597138s) [upload=7527]: Upload finished. err: <nil> Aug 7 23:48:09.882765252 (7h21m23.469821463s) [upload=7545]: Starting upload Aug 7 23:48:09.882860198 (7h21m23.46991642s) [upload=7545]: Done starting upload Aug 7 23:48:09.882935729 (7h21m23.469991941s) [upload=7545]: Uploading segments... Aug 7 23:48:09.882965635 (7h21m23.470021856s) [upload=7545]: Got next segment. Inline: true Aug 7 23:48:09.882969723 (7h21m23.470025944s) [upload=7545]: Waiting for error group managing segments... Aug 7 23:48:09.883002865 (7h21m23.470059086s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:09.883006481 (7h21m23.470062702s) [upload=7545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:09.883009045 (7h21m23.470065266s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:09.951272229 (7h21m23.538328451s) [upload=7545]: Done uploading segments... Aug 7 23:48:09.951286926 (7h21m23.538343148s) [upload=7545]: Upload finished. err: <nil> Aug 7 23:48:58.772502770 (7h22m12.359559011s) [upload=7557]: Starting upload Aug 7 23:48:58.772574805 (7h22m12.359631046s) [upload=7557]: Done starting upload Aug 7 23:48:58.772645527 (7h22m12.359701767s) [upload=7557]: Uploading segments... Aug 7 23:48:58.772695300 (7h22m12.359751541s) [upload=7557]: Got next segment. Inline: true Aug 7 23:48:58.772701031 (7h22m12.359757271s) [upload=7557]: Waiting for error group managing segments... Aug 7 23:48:58.772770249 (7h22m12.359826501s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:58.772780228 (7h22m12.359836469s) [upload=7557]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:58.772787010 (7h22m12.359843252s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:58.840436652 (7h22m12.427492893s) [upload=7557]: Done uploading segments... Aug 7 23:48:58.840462661 (7h22m12.427518891s) [upload=7557]: Upload finished. err: <nil> Aug 7 23:50:01.971851413 (7h23m15.558907624s) [upload=7574]: Starting upload Aug 7 23:50:01.971890696 (7h23m15.558946907s) [upload=7574]: Done starting upload Aug 7 23:50:01.971977749 (7h23m15.559033961s) [upload=7574]: Uploading segments... Aug 7 23:50:01.972022943 (7h23m15.559079154s) [upload=7574]: Got next segment. Inline: true Aug 7 23:50:01.972026850 (7h23m15.559083062s) [upload=7574]: Waiting for error group managing segments... Aug 7 23:50:01.972060975 (7h23m15.559117185s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:01.972064511 (7h23m15.559120712s) [upload=7574]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:01.972066685 (7h23m15.559122896s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:02.063709030 (7h23m15.650765241s) [upload=7574]: Done uploading segments... Aug 7 23:50:02.063723547 (7h23m15.650779768s) [upload=7574]: Upload finished. err: <nil> Aug 7 23:50:41.160166556 (7h23m54.747222777s) [upload=7588]: Starting upload Aug 7 23:50:41.160220555 (7h23m54.747276777s) [upload=7588]: Done starting upload Aug 7 23:50:41.160264659 (7h23m54.74732088s) [upload=7588]: Uploading segments... Aug 7 23:50:41.160295956 (7h23m54.747352177s) [upload=7588]: Got next segment. Inline: true Aug 7 23:50:41.160299753 (7h23m54.747355964s) [upload=7588]: Waiting for error group managing segments... Aug 7 23:50:41.160331633 (7h23m54.747387844s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:41.160335861 (7h23m54.747392072s) [upload=7588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:41.160338396 (7h23m54.747394617s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:41.222859735 (7h23m54.809915976s) [upload=7588]: Done uploading segments... Aug 7 23:50:41.222879913 (7h23m54.809936154s) [upload=7588]: Upload finished. err: <nil> Aug 7 23:51:17.131690986 (7h24m30.718747197s) [upload=7599]: Starting upload Aug 7 23:51:17.131738926 (7h24m30.718795147s) [upload=7599]: Done starting upload Aug 7 23:51:17.131792947 (7h24m30.718849178s) [upload=7599]: Uploading segments... Aug 7 23:51:17.131835896 (7h24m30.718892117s) [upload=7599]: Got next segment. Inline: true Aug 7 23:51:17.131842027 (7h24m30.718898239s) [upload=7599]: Waiting for error group managing segments... Aug 7 23:51:17.131883385 (7h24m30.718939627s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:17.131889146 (7h24m30.718945388s) [upload=7599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:17.132006205 (7h24m30.719062435s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:38.787433326 (7h24m52.374489567s) [upload=7599]: Done uploading segments... Aug 7 23:51:38.787452212 (7h24m52.374508453s) [upload=7599]: Upload finished. err: <nil> Aug 7 23:52:00.256204647 (7h25m13.843260858s) [upload=7610]: Starting upload Aug 7 23:52:00.256263547 (7h25m13.843319758s) [upload=7610]: Done starting upload Aug 7 23:52:00.256316195 (7h25m13.843372406s) [upload=7610]: Uploading segments... Aug 7 23:52:00.256345671 (7h25m13.843401881s) [upload=7610]: Got next segment. Inline: true Aug 7 23:52:00.256349758 (7h25m13.843405969s) [upload=7610]: Waiting for error group managing segments... Aug 7 23:52:00.256399992 (7h25m13.843456203s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:00.256404780 (7h25m13.843460991s) [upload=7610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:00.256407806 (7h25m13.843464017s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:30.237600164 (7h25m43.824656416s) [upload=7610]: Done uploading segments... Aug 7 23:52:30.237630862 (7h25m43.824687113s) [upload=7610]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:53.146251246 (7h26m6.733307456s) [upload=7626]: Starting upload Aug 7 23:52:53.146307339 (7h26m6.733363561s) [upload=7626]: Done starting upload Aug 7 23:52:53.146350962 (7h26m6.733407173s) [upload=7626]: Uploading segments... Aug 7 23:52:53.146411344 (7h26m6.733467565s) [upload=7626]: Got next segment. Inline: true Aug 7 23:52:53.146415271 (7h26m6.733471493s) [upload=7626]: Waiting for error group managing segments... Aug 7 23:52:53.146449966 (7h26m6.733506177s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:53.146454174 (7h26m6.733510395s) [upload=7626]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:53.146456709 (7h26m6.73351293s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:23.132536521 (7h26m36.719592742s) [upload=7626]: Done uploading segments... Aug 7 23:53:23.132560907 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001441521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [.........XXX.X.X.........................................XX.] duration: 1.423643444s FAIL (2025-08-07 23:54:21 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ading piece... Aug 7 23:51:17.347399839 (7h24m30.934456051s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347417763 (7h24m30.934473974s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347625520 (7h24m30.934681731s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347755754 (7h24m30.934811985s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347968761 (7h24m30.935024982s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348133560 (7h24m30.935189781s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348193100 (7h24m30.935249311s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348536501 (7h24m30.935592722s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348656385 (7h24m30.935712596s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348854374 (7h24m30.935910585s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348919236 (7h24m30.935975457s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.430583697 (7h24m31.017639918s) [upload=7600]: Next returned nil segment Aug 7 23:51:17.430596350 (7h24m31.017652571s) [upload=7600]: Waiting for error group managing segments... Aug 7 23:51:17.450496095 (7h24m31.037552306s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.457273115 (7h24m31.044329336s) [noise=true , address=45.91.116.11:4121 , piece_id=UI4HOUG4KLWC3LFGDVZ3C3IUMB2DGJP2QOTIOLWS5C2HUAMSATMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.458658320 (7h24m31.045714532s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.464324767 (7h24m31.051380978s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.469258046 (7h24m31.056314256s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.470547041 (7h24m31.057603252s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.482895480 (7h24m31.069951691s) [noise=true , address=104.36.82.59:4327 , piece_id=5AM7TVDV5ORD5WPHKEHOM652JETCJ6VH3JOJ6HPJTIOTABTYVELA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.503848269 (7h24m31.090904481s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.589438619 (7h24m31.17649484s) [noise=true , address=79.127.201.215:4343 , piece_id=6IVWBNSJNI4ZAEIWFTRYUHIHZ35YTTZF3FTEF7G76YEBUP2NBJFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.633863581 (7h24m31.220919802s) [noise=true , address=104.36.82.42:4801 , piece_id=ILNDV4CZCPEST6R4DUTSHQ5OJS2N4BLPRLDYIYNBTZKRR4RSCOFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.636493760 (7h24m31.22354997s) [noise=true , address=104.36.82.44:4904 , piece_id=TFXFZ53DGPOVVXTTZMRMTJR53DGXNHWUF43I7OCC4YNCRW4QUCKA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.637613389 (7h24m31.224669611s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.654845844 (7h24m31.241902065s) [noise=true , address=104.36.82.22:4524 , piece_id=66YIUJKSDV52T5NMMFHYCZH7L34UKH3RLIS5WZ3VWRBL3KEE7UWQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853873712 (7h24m31.440929933s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853888850 (7h24m31.440945061s) [seg_pos={0 0}, upload=7600]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:17.853987013 (7h24m31.441043234s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:17.854295209 (7h24m31.44135142s) [noise=true , address=104.36.82.46:4499 , piece_id=6OBTEQJKT4AJRGUT27EYWNLG4E2TKBCQBRGXTTEOU4SJY2SWUTFQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection Aug 7 23:51:17.854338279 (7h24m31.44139449s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zkqEMSKx7zSHPFNZ51b5TLzL2cFQL8VyxAG14Lg1pT8zAu3tc): protocol: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection Aug 7 23:51:17.854399554 (7h24m31.441455765s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gPjPGuueZnizLT2QvJN9y7riwKiW5TWMkQtmGGPP7VZvkNoB7): protocol: read tcp 5.161.89.252:35428->104.36.82.15:4288: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:51:17.854419941 (7h24m31.441476162s) [seg_pos={0 0}, upload=7600]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:17.854472530 (7h24m31.441528751s) [upload=7600]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:17.854476738 (7h24m31.441532959s) [upload=7600]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:17.900817215 (7h24m31.487873465s) [upload=7600]: Done uploading segments... Aug 7 23:51:17.900834827 (7h24m31.487891079s) [upload=7600]: Upload finished. err: <nil> Aug 7 23:52:11.382772324 (7h25m24.969828534s) [upload=7616]: Starting upload Aug 7 23:52:11.382828459 (7h25m24.96988468s) [upload=7616]: Done starting upload Aug 7 23:52:11.382871529 (7h25m24.96992774s) [upload=7616]: Uploading segments... Aug 7 23:52:11.383053639 (7h25m24.97010986s) [upload=7616]: Got next segment. Inline: false Aug 7 23:52:11.383080680 (7h25m24.970136891s) [upload=7616]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:11.383084406 (7h25m24.970140617s) [upload=7616]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:11.371970391 (7h26m24.959026631s) [upload=7616]: Next segment err: context deadline exceeded Aug 7 23:53:11.372056972 (7h26m24.959113223s) [upload=7616]: Done uploading segments... Aug 7 23:53:11.372090856 (7h26m24.959147096s) [upload=7616]: Upload finished. err: context deadline exceeded Aug 7 23:53:11.462534524 (7h26m25.049590775s) [upload=7633]: Starting upload Aug 7 23:53:11.462662182 (7h26m25.049718423s) [upload=7633]: Done starting upload Aug 7 23:53:11.462730148 (7h26m25.0497864s) [upload=7633]: Uploading segments... Aug 7 23:53:11.463218160 (7h26m25.050274451s) [upload=7633]: Got next segment. Inline: false Aug 7 23:53:11.463267292 (7h26m25.050323533s) [upload=7633]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.463276689 (7h26m25.050332931s) [upload=7633]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:11.448954917 (7h27m25.036011128s) [upload=7633]: Next segment err: context deadline exceeded Aug 7 23:54:11.449030117 (7h27m25.036086328s) [upload=7633]: Done uploading segments... Aug 7 23:54:11.449047229 (7h27m25.03610344s) [upload=7633]: Upload finished. err: context deadline exceeded Duration: 1.423672139s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...XXXXXX.X.X...........................................X.X.] duration: 2.011533337s FAIL (2025-08-07 23:54:16 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.011552763s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 153.8332ms FAIL (2025-08-07 23:54:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 153.883815ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [......X.XXXXX..X............................................] duration: 199.462919ms FAIL (2025-08-07 23:54:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 199.48029ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 874.830401ms FAIL (2025-08-07 23:54:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 874.854346ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [..........XXX.X.X.........................................XX] duration: 1m0.00046619s FAIL (2025-08-07 23:54:11 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== ading piece... Aug 7 23:51:17.347399839 (7h24m30.934456051s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347417763 (7h24m30.934473974s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347625520 (7h24m30.934681731s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347755754 (7h24m30.934811985s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347968761 (7h24m30.935024982s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348133560 (7h24m30.935189781s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348193100 (7h24m30.935249311s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348536501 (7h24m30.935592722s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348656385 (7h24m30.935712596s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348854374 (7h24m30.935910585s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348919236 (7h24m30.935975457s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.430583697 (7h24m31.017639918s) [upload=7600]: Next returned nil segment Aug 7 23:51:17.430596350 (7h24m31.017652571s) [upload=7600]: Waiting for error group managing segments... Aug 7 23:51:17.450496095 (7h24m31.037552306s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.457273115 (7h24m31.044329336s) [noise=true , address=45.91.116.11:4121 , piece_id=UI4HOUG4KLWC3LFGDVZ3C3IUMB2DGJP2QOTIOLWS5C2HUAMSATMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.458658320 (7h24m31.045714532s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.464324767 (7h24m31.051380978s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.469258046 (7h24m31.056314256s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.470547041 (7h24m31.057603252s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.482895480 (7h24m31.069951691s) [noise=true , address=104.36.82.59:4327 , piece_id=5AM7TVDV5ORD5WPHKEHOM652JETCJ6VH3JOJ6HPJTIOTABTYVELA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.503848269 (7h24m31.090904481s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.589438619 (7h24m31.17649484s) [noise=true , address=79.127.201.215:4343 , piece_id=6IVWBNSJNI4ZAEIWFTRYUHIHZ35YTTZF3FTEF7G76YEBUP2NBJFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.633863581 (7h24m31.220919802s) [noise=true , address=104.36.82.42:4801 , piece_id=ILNDV4CZCPEST6R4DUTSHQ5OJS2N4BLPRLDYIYNBTZKRR4RSCOFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.636493760 (7h24m31.22354997s) [noise=true , address=104.36.82.44:4904 , piece_id=TFXFZ53DGPOVVXTTZMRMTJR53DGXNHWUF43I7OCC4YNCRW4QUCKA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.637613389 (7h24m31.224669611s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.654845844 (7h24m31.241902065s) [noise=true , address=104.36.82.22:4524 , piece_id=66YIUJKSDV52T5NMMFHYCZH7L34UKH3RLIS5WZ3VWRBL3KEE7UWQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853873712 (7h24m31.440929933s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853888850 (7h24m31.440945061s) [seg_pos={0 0}, upload=7600]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:17.853987013 (7h24m31.441043234s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:17.854295209 (7h24m31.44135142s) [noise=true , address=104.36.82.46:4499 , piece_id=6OBTEQJKT4AJRGUT27EYWNLG4E2TKBCQBRGXTTEOU4SJY2SWUTFQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection Aug 7 23:51:17.854338279 (7h24m31.44139449s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zkqEMSKx7zSHPFNZ51b5TLzL2cFQL8VyxAG14Lg1pT8zAu3tc): protocol: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection Aug 7 23:51:17.854399554 (7h24m31.441455765s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gPjPGuueZnizLT2QvJN9y7riwKiW5TWMkQtmGGPP7VZvkNoB7): protocol: read tcp 5.161.89.252:35428->104.36.82.15:4288: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:51:17.854419941 (7h24m31.441476162s) [seg_pos={0 0}, upload=7600]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:17.854472530 (7h24m31.441528751s) [upload=7600]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:17.854476738 (7h24m31.441532959s) [upload=7600]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:17.900817215 (7h24m31.487873465s) [upload=7600]: Done uploading segments... Aug 7 23:51:17.900834827 (7h24m31.487891079s) [upload=7600]: Upload finished. err: <nil> Aug 7 23:52:11.382772324 (7h25m24.969828534s) [upload=7616]: Starting upload Aug 7 23:52:11.382828459 (7h25m24.96988468s) [upload=7616]: Done starting upload Aug 7 23:52:11.382871529 (7h25m24.96992774s) [upload=7616]: Uploading segments... Aug 7 23:52:11.383053639 (7h25m24.97010986s) [upload=7616]: Got next segment. Inline: false Aug 7 23:52:11.383080680 (7h25m24.970136891s) [upload=7616]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:11.383084406 (7h25m24.970140617s) [upload=7616]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:11.371970391 (7h26m24.959026631s) [upload=7616]: Next segment err: context deadline exceeded Aug 7 23:53:11.372056972 (7h26m24.959113223s) [upload=7616]: Done uploading segments... Aug 7 23:53:11.372090856 (7h26m24.959147096s) [upload=7616]: Upload finished. err: context deadline exceeded Aug 7 23:53:11.462534524 (7h26m25.049590775s) [upload=7633]: Starting upload Aug 7 23:53:11.462662182 (7h26m25.049718423s) [upload=7633]: Done starting upload Aug 7 23:53:11.462730148 (7h26m25.0497864s) [upload=7633]: Uploading segments... Aug 7 23:53:11.463218160 (7h26m25.050274451s) [upload=7633]: Got next segment. Inline: false Aug 7 23:53:11.463267292 (7h26m25.050323533s) [upload=7633]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.463276689 (7h26m25.050332931s) [upload=7633]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:54:11.448954917 (7h27m25.036011128s) [upload=7633]: Next segment err: context deadline exceeded Aug 7 23:54:11.449030117 (7h27m25.036086328s) [upload=7633]: Done uploading segments... Aug 7 23:54:11.449047229 (7h27m25.03610344s) [upload=7633]: Upload finished. err: context deadline exceeded Duration: 1m0.000480166s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [....XXXXXX.X...............................................X] duration: 1m0.001179591s FAIL (2025-08-07 23:54:11 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :45:31.386895707 (7h18m44.973951958s) [upload=7501]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:31.448582950 (7h18m45.035639191s) [upload=7501]: Done uploading segments... Aug 7 23:45:31.448602987 (7h18m45.035659229s) [upload=7501]: Upload finished. err: <nil> Aug 7 23:46:05.500265801 (7h19m19.087322041s) [upload=7512]: Starting upload Aug 7 23:46:05.500376477 (7h19m19.087432728s) [upload=7512]: Done starting upload Aug 7 23:46:05.500456777 (7h19m19.087513028s) [upload=7512]: Uploading segments... Aug 7 23:46:05.500516349 (7h19m19.08757266s) [upload=7512]: Got next segment. Inline: true Aug 7 23:46:05.500526007 (7h19m19.087582288s) [upload=7512]: Waiting for error group managing segments... Aug 7 23:46:05.500594884 (7h19m19.087651126s) [upload=7512]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:05.500603801 (7h19m19.087660053s) [upload=7512]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:05.500610143 (7h19m19.087666395s) [upload=7512]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:05.713026270 (7h19m19.300082522s) [upload=7512]: Done uploading segments... Aug 7 23:46:05.713042230 (7h19m19.300098472s) [upload=7512]: Upload finished. err: <nil> Aug 7 23:46:59.827069649 (7h20m13.41412587s) [upload=7526]: Starting upload Aug 7 23:46:59.827112839 (7h20m13.414169051s) [upload=7526]: Done starting upload Aug 7 23:46:59.827158284 (7h20m13.414214506s) [upload=7526]: Uploading segments... Aug 7 23:46:59.827184925 (7h20m13.414241135s) [upload=7526]: Got next segment. Inline: true Aug 7 23:46:59.827188771 (7h20m13.414244992s) [upload=7526]: Waiting for error group managing segments... Aug 7 23:46:59.827219097 (7h20m13.414275308s) [upload=7526]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:59.827222835 (7h20m13.414279056s) [upload=7526]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:59.827225119 (7h20m13.414281329s) [upload=7526]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:59.879507639 (7h20m13.46656385s) [upload=7526]: Done uploading segments... Aug 7 23:46:59.879523058 (7h20m13.466579269s) [upload=7526]: Upload finished. err: <nil> Aug 7 23:47:39.924312697 (7h20m53.511368928s) [upload=7536]: Starting upload Aug 7 23:47:39.924413405 (7h20m53.511469646s) [upload=7536]: Done starting upload Aug 7 23:47:39.924491079 (7h20m53.51154731s) [upload=7536]: Uploading segments... Aug 7 23:47:39.924700320 (7h20m53.511756551s) [upload=7536]: Got next segment. Inline: true Aug 7 23:47:39.924711842 (7h20m53.511768063s) [upload=7536]: Waiting for error group managing segments... Aug 7 23:47:39.924828248 (7h20m53.51188448s) [upload=7536]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:39.924836915 (7h20m53.511893156s) [upload=7536]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:39.924842024 (7h20m53.511898255s) [upload=7536]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.001205869s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....XXXXXX.X.X...........................................X.X] duration: 30.001101184s FAIL (2025-08-07 23:54:09 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001122374s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X.XX.X....X.X..........................................X.] duration: 619.841882ms FAIL (2025-08-07 23:54:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 619.860917ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [......XXXXX..X............................................XX] duration: 1m0.000656916s FAIL (2025-08-07 23:54:00 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000678456s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [....XXXXX.XX............................................XXXX] duration: 30.001119922s FAIL (2025-08-07 23:53:58 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:42.664256383 (7h18m56.251312604s) [upload=7507]: Done uploading segments... Aug 7 23:45:42.664275208 (7h18m56.251331429s) [upload=7507]: Upload finished. err: <nil> Aug 7 23:46:08.906811448 (7h19m22.493867688s) [upload=7514]: Starting upload Aug 7 23:46:08.906885386 (7h19m22.493941627s) [upload=7514]: Done starting upload Aug 7 23:46:08.906964935 (7h19m22.494021175s) [upload=7514]: Uploading segments... Aug 7 23:46:08.906997987 (7h19m22.494054228s) [upload=7514]: Got next segment. Inline: true Aug 7 23:46:08.907003197 (7h19m22.494059437s) [upload=7514]: Waiting for error group managing segments... Aug 7 23:46:08.907046577 (7h19m22.494102807s) [upload=7514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:08.907051767 (7h19m22.494108007s) [upload=7514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:08.907055173 (7h19m22.494111414s) [upload=7514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:08.965075092 (7h19m22.552131313s) [upload=7514]: Done uploading segments... Aug 7 23:46:08.965095970 (7h19m22.552152191s) [upload=7514]: Upload finished. err: <nil> Aug 7 23:47:00.017908282 (7h20m13.604964493s) [upload=7527]: Starting upload Aug 7 23:47:00.017955210 (7h20m13.605011411s) [upload=7527]: Done starting upload Aug 7 23:47:00.017998241 (7h20m13.605054452s) [upload=7527]: Uploading segments... Aug 7 23:47:00.018026292 (7h20m13.605082503s) [upload=7527]: Got next segment. Inline: true Aug 7 23:47:00.018029438 (7h20m13.60508565s) [upload=7527]: Waiting for error group managing segments... Aug 7 23:47:00.018058453 (7h20m13.605114664s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:00.018062390 (7h20m13.605118601s) [upload=7527]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:00.018064304 (7h20m13.605120515s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:00.075526210 (7h20m13.662582431s) [upload=7527]: Done uploading segments... Aug 7 23:47:00.075540917 (7h20m13.662597138s) [upload=7527]: Upload finished. err: <nil> Aug 7 23:48:09.882765252 (7h21m23.469821463s) [upload=7545]: Starting upload Aug 7 23:48:09.882860198 (7h21m23.46991642s) [upload=7545]: Done starting upload Aug 7 23:48:09.882935729 (7h21m23.469991941s) [upload=7545]: Uploading segments... Aug 7 23:48:09.882965635 (7h21m23.470021856s) [upload=7545]: Got next segment. Inline: true Aug 7 23:48:09.882969723 (7h21m23.470025944s) [upload=7545]: Waiting for error group managing segments... Aug 7 23:48:09.883002865 (7h21m23.470059086s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:09.883006481 (7h21m23.470062702s) [upload=7545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:09.883009045 (7h21m23.470065266s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:09.951272229 (7h21m23.538328451s) [upload=7545]: Done uploading segments... Aug 7 23:48:09.951286926 (7h21m23.538343148s) [upload=7545]: Upload finished. err: <nil> Aug 7 23:48:58.772502770 (7h22m12.359559011s) [upload=7557]: Starting upload Aug 7 23:48:58.772574805 (7h22m12.359631046s) [upload=7557]: Done starting upload Aug 7 23:48:58.772645527 (7h22m12.359701767s) [upload=7557]: Uploading segments... Aug 7 23:48:58.772695300 (7h22m12.359751541s) [upload=7557]: Got next segment. Inline: true Aug 7 23:48:58.772701031 (7h22m12.359757271s) [upload=7557]: Waiting for error group managing segments... Aug 7 23:48:58.772770249 (7h22m12.359826501s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:58.772780228 (7h22m12.359836469s) [upload=7557]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:58.772787010 (7h22m12.359843252s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:58.840436652 (7h22m12.427492893s) [upload=7557]: Done uploading segments... Aug 7 23:48:58.840462661 (7h22m12.427518891s) [upload=7557]: Upload finished. err: <nil> Aug 7 23:50:01.971851413 (7h23m15.558907624s) [upload=7574]: Starting upload Aug 7 23:50:01.971890696 (7h23m15.558946907s) [upload=7574]: Done starting upload Aug 7 23:50:01.971977749 (7h23m15.559033961s) [upload=7574]: Uploading segments... Aug 7 23:50:01.972022943 (7h23m15.559079154s) [upload=7574]: Got next segment. Inline: true Aug 7 23:50:01.972026850 (7h23m15.559083062s) [upload=7574]: Waiting for error group managing segments... Aug 7 23:50:01.972060975 (7h23m15.559117185s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:01.972064511 (7h23m15.559120712s) [upload=7574]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:01.972066685 (7h23m15.559122896s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:02.063709030 (7h23m15.650765241s) [upload=7574]: Done uploading segments... Aug 7 23:50:02.063723547 (7h23m15.650779768s) [upload=7574]: Upload finished. err: <nil> Aug 7 23:50:41.160166556 (7h23m54.747222777s) [upload=7588]: Starting upload Aug 7 23:50:41.160220555 (7h23m54.747276777s) [upload=7588]: Done starting upload Aug 7 23:50:41.160264659 (7h23m54.74732088s) [upload=7588]: Uploading segments... Aug 7 23:50:41.160295956 (7h23m54.747352177s) [upload=7588]: Got next segment. Inline: true Aug 7 23:50:41.160299753 (7h23m54.747355964s) [upload=7588]: Waiting for error group managing segments... Aug 7 23:50:41.160331633 (7h23m54.747387844s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:41.160335861 (7h23m54.747392072s) [upload=7588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:41.160338396 (7h23m54.747394617s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:41.222859735 (7h23m54.809915976s) [upload=7588]: Done uploading segments... Aug 7 23:50:41.222879913 (7h23m54.809936154s) [upload=7588]: Upload finished. err: <nil> Aug 7 23:51:17.131690986 (7h24m30.718747197s) [upload=7599]: Starting upload Aug 7 23:51:17.131738926 (7h24m30.718795147s) [upload=7599]: Done starting upload Aug 7 23:51:17.131792947 (7h24m30.718849178s) [upload=7599]: Uploading segments... Aug 7 23:51:17.131835896 (7h24m30.718892117s) [upload=7599]: Got next segment. Inline: true Aug 7 23:51:17.131842027 (7h24m30.718898239s) [upload=7599]: Waiting for error group managing segments... Aug 7 23:51:17.131883385 (7h24m30.718939627s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:17.131889146 (7h24m30.718945388s) [upload=7599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:17.132006205 (7h24m30.719062435s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:38.787433326 (7h24m52.374489567s) [upload=7599]: Done uploading segments... Aug 7 23:51:38.787452212 (7h24m52.374508453s) [upload=7599]: Upload finished. err: <nil> Aug 7 23:52:00.256204647 (7h25m13.843260858s) [upload=7610]: Starting upload Aug 7 23:52:00.256263547 (7h25m13.843319758s) [upload=7610]: Done starting upload Aug 7 23:52:00.256316195 (7h25m13.843372406s) [upload=7610]: Uploading segments... Aug 7 23:52:00.256345671 (7h25m13.843401881s) [upload=7610]: Got next segment. Inline: true Aug 7 23:52:00.256349758 (7h25m13.843405969s) [upload=7610]: Waiting for error group managing segments... Aug 7 23:52:00.256399992 (7h25m13.843456203s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:00.256404780 (7h25m13.843460991s) [upload=7610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:00.256407806 (7h25m13.843464017s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:30.237600164 (7h25m43.824656416s) [upload=7610]: Done uploading segments... Aug 7 23:52:30.237630862 (7h25m43.824687113s) [upload=7610]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:53.146251246 (7h26m6.733307456s) [upload=7626]: Starting upload Aug 7 23:52:53.146307339 (7h26m6.733363561s) [upload=7626]: Done starting upload Aug 7 23:52:53.146350962 (7h26m6.733407173s) [upload=7626]: Uploading segments... Aug 7 23:52:53.146411344 (7h26m6.733467565s) [upload=7626]: Got next segment. Inline: true Aug 7 23:52:53.146415271 (7h26m6.733471493s) [upload=7626]: Waiting for error group managing segments... Aug 7 23:52:53.146449966 (7h26m6.733506177s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:53.146454174 (7h26m6.733510395s) [upload=7626]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:53.146456709 (7h26m6.73351293s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:23.132536521 (7h26m36.719592742s) [upload=7626]: Done uploading segments... Aug 7 23:53:23.132560907 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.001141061s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X.XX.X....X.X..........................................X] duration: 30.000347718s FAIL (2025-08-07 23:53:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000366883s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 458.21592ms FAIL (2025-08-07 23:53:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 458.243251ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.......X.XXXXX..X...........................................] duration: 215.127128ms FAIL (2025-08-07 23:53:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 215.148538ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X....XXX.X......................................X] duration: 30.000543878s FAIL (2025-08-07 23:53:32 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== [upload=7594]: Upload finished. err: <nil> Aug 7 23:52:06.728352003 (7h25m20.315408205s) [upload=7613]: Starting upload Aug 7 23:52:06.728452502 (7h25m20.315508713s) [upload=7613]: Done starting upload Aug 7 23:52:06.728510219 (7h25m20.31556647s) [upload=7613]: Uploading segments... Aug 7 23:52:06.728794470 (7h25m20.315850712s) [upload=7613]: Got next segment. Inline: false Aug 7 23:52:06.728839153 (7h25m20.315895395s) [upload=7613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:06.728845906 (7h25m20.315902147s) [upload=7613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:52:06.752678421 (7h25m20.339734662s) [seg_pos={0 0}, upload=7613]: Begin upload segment... Aug 7 23:52:06.752737542 (7h25m20.339793782s) [seg_pos={0 0}, upload=7613]: Done begin upload segment. Aug 7 23:52:06.752769762 (7h25m20.339826003s) [noise=true , address=77.90.31.12:4590 , piece_id=PQUGWXWL3BFAO2QOB3TN7ERFO7ZZXVYTEX2BJOT2Y5APPFPICJFQ, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.752823492 (7h25m20.339879713s) [noise=true , address=45.91.116.8:4045 , piece_id=JPMO37KLHKIQUAQBORBK64FXGHE4ZOVTGR4CEESADPJVIRIW2QFA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.753236552 (7h25m20.340292804s) [noise=true , address=77.90.31.14:4661 , piece_id=LLFIZYQUTYSDQJ7EQBAEPJ3Z3S2FRBVEIIOV4B5IQYFVVBG4FZXA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.753261680 (7h25m20.34031792s) [noise=true , address=104.36.82.4:4801 , piece_id=E5YH3CCHLGC3PZX6YXSVJ54YIWVMGL42IOSNOVFUWAKVVXKFTPUA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.753565126 (7h25m20.340621337s) [noise=true , address=45.91.116.7:4010 , piece_id=36CQJILODTNMU66BQ6U7QEM6UZUTKAX2RHHAWKBHYU6ZJAXO4IGA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.753655274 (7h25m20.340711516s) [noise=true , address=45.91.116.5:4299 , piece_id=XZKLTHUUSYNYNCQB7CZM3L2ZSDMZDCGGCO4NV4TPICMZM7BQKVGQ, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.753854125 (7h25m20.340910337s) [noise=true , address=104.36.82.7:4912 , piece_id=OUFQD5IYM422I32OHZA2XK5HGLX3IWO3QQIGB3OQPZ53ZMKU5KZA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.754018442 (7h25m20.341074683s) [noise=true , address=45.91.116.9:4263 , piece_id=YWUCFNNUEOFPDOF3CD2DGASWYKY3OEHDR4G34VSXBN3UBPGQYVAQ, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.754133126 (7h25m20.341189337s) [noise=true , address=77.90.31.18:4806 , piece_id=UTIDGJ6FJSQSM4NMZMM6EIAOXXQGDWKR5FTEUCZZM4JUDL4H7IBQ, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.754397169 (7h25m20.341453411s) [noise=true , address=45.91.116.6:4975 , piece_id=RFDVEUP3YNCDZCKGGBFNVLQ2RAVRBF5ZR2MH6NZDINNHLMVC34FA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.754462500 (7h25m20.341518721s) [noise=true , address=45.91.116.10:4081 , piece_id=2VCAS4RZ645LBNNB3CRUI6BIWKMT7DANB25XFN3PN2S3OPBR5BMA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.754745610 (7h25m20.341801821s) [noise=true , address=104.36.82.9:4007 , piece_id=IYNBWMWGJJHXWCFY2UNTN6BOZUZQCEG5HMDP65AVBNJZXBDL25CA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.754756841 (7h25m20.341813082s) [noise=true , address=104.36.82.4:4803 , piece_id=APSBYVSPJ3BLZ36FPEZQF3NO2MAQRGCBIOT24KAA47I5N2O3ZI2Q, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.755031362 (7h25m20.342087574s) [noise=true , address=45.91.116.11:4119 , piece_id=JVSG7QVEL5PFH25WQDGUROYOLK256J7GCEMA4TOSSVXIVI3ZNUYA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.755131800 (7h25m20.342188041s) [noise=true , address=45.91.116.4:4938 , piece_id=YNZBEVYM5S7VV54MSNF26HMGU7JFHTPA5TIOI42C5ZCOZUDGS3SA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.755569387 (7h25m20.342625638s) [noise=true , address=45.91.116.13:4153 , piece_id=77TRA6PWDU6D5NTRN27BC2PEUNRPIW7ONITHPM56QQ4HWYZ5V5NA, seg_pos={0 0}, upload=7613]: Uploading piece... Aug 7 23:52:06.841187318 (7h25m20.42824353s) [upload=7613]: Next returned nil segment Aug 7 23:52:06.841207075 (7h25m20.428263286s) [upload=7613]: Waiting for error group managing segments... Aug 7 23:52:06.849202168 (7h25m20.43625841s) [noise=true , address=45.91.116.4:4938 , piece_id=YNZBEVYM5S7VV54MSNF26HMGU7JFHTPA5TIOI42C5ZCOZUDGS3SA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.849781229 (7h25m20.436837439s) [noise=true , address=77.90.31.12:4590 , piece_id=PQUGWXWL3BFAO2QOB3TN7ERFO7ZZXVYTEX2BJOT2Y5APPFPICJFQ, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.850187878 (7h25m20.43724409s) [noise=true , address=45.91.116.10:4081 , piece_id=2VCAS4RZ645LBNNB3CRUI6BIWKMT7DANB25XFN3PN2S3OPBR5BMA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.850510370 (7h25m20.437566621s) [noise=true , address=45.91.116.13:4153 , piece_id=77TRA6PWDU6D5NTRN27BC2PEUNRPIW7ONITHPM56QQ4HWYZ5V5NA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.850948047 (7h25m20.438004258s) [noise=true , address=45.91.116.9:4263 , piece_id=YWUCFNNUEOFPDOF3CD2DGASWYKY3OEHDR4G34VSXBN3UBPGQYVAQ, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.851335932 (7h25m20.438392142s) [noise=true , address=45.91.116.8:4045 , piece_id=JPMO37KLHKIQUAQBORBK64FXGHE4ZOVTGR4CEESADPJVIRIW2QFA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.851618539 (7h25m20.438674749s) [noise=true , address=45.91.116.11:4119 , piece_id=JVSG7QVEL5PFH25WQDGUROYOLK256J7GCEMA4TOSSVXIVI3ZNUYA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.851736418 (7h25m20.43879267s) [noise=true , address=45.91.116.7:4010 , piece_id=36CQJILODTNMU66BQ6U7QEM6UZUTKAX2RHHAWKBHYU6ZJAXO4IGA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.851936792 (7h25m20.438993003s) [noise=true , address=77.90.31.18:4806 , piece_id=UTIDGJ6FJSQSM4NMZMM6EIAOXXQGDWKR5FTEUCZZM4JUDL4H7IBQ, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.853958856 (7h25m20.441015098s) [noise=true , address=45.91.116.5:4299 , piece_id=XZKLTHUUSYNYNCQB7CZM3L2ZSDMZDCGGCO4NV4TPICMZM7BQKVGQ, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.854659484 (7h25m20.441715696s) [noise=true , address=45.91.116.6:4975 , piece_id=RFDVEUP3YNCDZCKGGBFNVLQ2RAVRBF5ZR2MH6NZDINNHLMVC34FA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.863188123 (7h25m20.450244334s) [noise=true , address=104.36.82.7:4912 , piece_id=OUFQD5IYM422I32OHZA2XK5HGLX3IWO3QQIGB3OQPZ53ZMKU5KZA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.864977654 (7h25m20.452033895s) [noise=true , address=104.36.82.4:4803 , piece_id=APSBYVSPJ3BLZ36FPEZQF3NO2MAQRGCBIOT24KAA47I5N2O3ZI2Q, seg_pos={0 0}, upload=7613]: Done uploading piece. err: <nil> Aug 7 23:52:06.864998413 (7h25m20.452054654s) [seg_pos={0 0}, upload=7613]: Segment reached optimal threshold of 13 pieces. Aug 7 23:52:06.865176945 (7h25m20.452233157s) [noise=true , address=77.90.31.14:4661 , piece_id=LLFIZYQUTYSDQJ7EQBAEPJ3Z3S2FRBVEIIOV4B5IQYFVVBG4FZXA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Aug 7 23:52:06.865299825 (7h25m20.452356066s) [noise=true , address=104.36.82.4:4801 , piece_id=E5YH3CCHLGC3PZX6YXSVJ54YIWVMGL42IOSNOVFUWAKVVXKFTPUA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 7 23:52:06.865339630 (7h25m20.45239584s) [noise=true , address=104.36.82.9:4007 , piece_id=IYNBWMWGJJHXWCFY2UNTN6BOZUZQCEG5HMDP65AVBNJZXBDL25CA, seg_pos={0 0}, upload=7613]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:8386->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:8386->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:8386->104.36.82.9:4007: use of closed network connection Aug 7 23:52:06.865360338 (7h25m20.45241654s) [seg_pos={0 0}, upload=7613]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:52:06.865450757 (7h25m20.452506968s) [upload=7613]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:52:06.865454935 (7h25m20.452511146s) [upload=7613]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:07.015091480 (7h25m20.602147711s) [upload=7613]: Done uploading segments... Aug 7 23:52:07.015111889 (7h25m20.60216813s) [upload=7613]: Upload finished. err: <nil> Aug 7 23:53:02.405645052 (7h26m15.992701273s) [upload=7627]: Starting upload Aug 7 23:53:02.405704283 (7h26m15.992760504s) [upload=7627]: Done starting upload Aug 7 23:53:02.405806533 (7h26m15.992862744s) [upload=7627]: Uploading segments... Aug 7 23:53:02.406021685 (7h26m15.993077906s) [upload=7627]: Got next segment. Inline: false Aug 7 23:53:02.406061369 (7h26m15.99311764s) [upload=7627]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:02.406068102 (7h26m15.993124323s) [upload=7627]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:32.387466061 (7h26m45.974522302s) [upload=7627]: Next segment err: context deadline exceeded Aug 7 23:53:32.387608347 (7h26m45.974664588s) [upload=7627]: Done uploading segments... Aug 7 23:53:32.387634907 (7h26m45.974691158s) [upload=7627]: Upload finished. err: context deadline exceeded Duration: 30.000638194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.....XXXXX.XX............................................XXX] duration: 30.000953405s FAIL (2025-08-07 23:53:23 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ) [upload=7488]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:35.894656303 (7h17m49.481712545s) [upload=7488]: Done uploading segments... Aug 7 23:44:35.894676211 (7h17m49.481732452s) [upload=7488]: Upload finished. err: <nil> Aug 7 23:45:42.609270547 (7h18m56.196326759s) [upload=7507]: Starting upload Aug 7 23:45:42.609334396 (7h18m56.196390617s) [upload=7507]: Done starting upload Aug 7 23:45:42.609380774 (7h18m56.196436984s) [upload=7507]: Uploading segments... Aug 7 23:45:42.609412683 (7h18m56.196468914s) [upload=7507]: Got next segment. Inline: true Aug 7 23:45:42.609418174 (7h18m56.196474385s) [upload=7507]: Waiting for error group managing segments... Aug 7 23:45:42.609455492 (7h18m56.196511713s) [upload=7507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:45:42.609459600 (7h18m56.196515811s) [upload=7507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:45:42.609462095 (7h18m56.196518316s) [upload=7507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:42.664256383 (7h18m56.251312604s) [upload=7507]: Done uploading segments... Aug 7 23:45:42.664275208 (7h18m56.251331429s) [upload=7507]: Upload finished. err: <nil> Aug 7 23:46:08.906811448 (7h19m22.493867688s) [upload=7514]: Starting upload Aug 7 23:46:08.906885386 (7h19m22.493941627s) [upload=7514]: Done starting upload Aug 7 23:46:08.906964935 (7h19m22.494021175s) [upload=7514]: Uploading segments... Aug 7 23:46:08.906997987 (7h19m22.494054228s) [upload=7514]: Got next segment. Inline: true Aug 7 23:46:08.907003197 (7h19m22.494059437s) [upload=7514]: Waiting for error group managing segments... Aug 7 23:46:08.907046577 (7h19m22.494102807s) [upload=7514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:08.907051767 (7h19m22.494108007s) [upload=7514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:08.907055173 (7h19m22.494111414s) [upload=7514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:08.965075092 (7h19m22.552131313s) [upload=7514]: Done uploading segments... Aug 7 23:46:08.965095970 (7h19m22.552152191s) [upload=7514]: Upload finished. err: <nil> Aug 7 23:47:00.017908282 (7h20m13.604964493s) [upload=7527]: Starting upload Aug 7 23:47:00.017955210 (7h20m13.605011411s) [upload=7527]: Done starting upload Aug 7 23:47:00.017998241 (7h20m13.605054452s) [upload=7527]: Uploading segments... Aug 7 23:47:00.018026292 (7h20m13.605082503s) [upload=7527]: Got next segment. Inline: true Aug 7 23:47:00.018029438 (7h20m13.60508565s) [upload=7527]: Waiting for error group managing segments... Aug 7 23:47:00.018058453 (7h20m13.605114664s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:00.018062390 (7h20m13.605118601s) [upload=7527]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:00.018064304 (7h20m13.605120515s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:00.075526210 (7h20m13.662582431s) [upload=7527]: Done uploading segments... Aug 7 23:47:00.075540917 (7h20m13.662597138s) [upload=7527]: Upload finished. err: <nil> Aug 7 23:48:09.882765252 (7h21m23.469821463s) [upload=7545]: Starting upload Aug 7 23:48:09.882860198 (7h21m23.46991642s) [upload=7545]: Done starting upload Aug 7 23:48:09.882935729 (7h21m23.469991941s) [upload=7545]: Uploading segments... Aug 7 23:48:09.882965635 (7h21m23.470021856s) [upload=7545]: Got next segment. Inline: true Aug 7 23:48:09.882969723 (7h21m23.470025944s) [upload=7545]: Waiting for error group managing segments... Aug 7 23:48:09.883002865 (7h21m23.470059086s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:09.883006481 (7h21m23.470062702s) [upload=7545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:09.883009045 (7h21m23.470065266s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:09.951272229 (7h21m23.538328451s) [upload=7545]: Done uploading segments... Aug 7 23:48:09.951286926 (7h21m23.538343148s) [upload=7545]: Upload finished. err: <nil> Aug 7 23:48:58.772502770 (7h22m12.359559011s) [upload=7557]: Starting upload Aug 7 23:48:58.772574805 (7h22m12.359631046s) [upload=7557]: Done starting upload Aug 7 23:48:58.772645527 (7h22m12.359701767s) [upload=7557]: Uploading segments... Aug 7 23:48:58.772695300 (7h22m12.359751541s) [upload=7557]: Got next segment. Inline: true Aug 7 23:48:58.772701031 (7h22m12.359757271s) [upload=7557]: Waiting for error group managing segments... Aug 7 23:48:58.772770249 (7h22m12.359826501s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:58.772780228 (7h22m12.359836469s) [upload=7557]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:58.772787010 (7h22m12.359843252s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:58.840436652 (7h22m12.427492893s) [upload=7557]: Done uploading segments... Aug 7 23:48:58.840462661 (7h22m12.427518891s) [upload=7557]: Upload finished. err: <nil> Aug 7 23:50:01.971851413 (7h23m15.558907624s) [upload=7574]: Starting upload Aug 7 23:50:01.971890696 (7h23m15.558946907s) [upload=7574]: Done starting upload Aug 7 23:50:01.971977749 (7h23m15.559033961s) [upload=7574]: Uploading segments... Aug 7 23:50:01.972022943 (7h23m15.559079154s) [upload=7574]: Got next segment. Inline: true Aug 7 23:50:01.972026850 (7h23m15.559083062s) [upload=7574]: Waiting for error group managing segments... Aug 7 23:50:01.972060975 (7h23m15.559117185s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:01.972064511 (7h23m15.559120712s) [upload=7574]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:01.972066685 (7h23m15.559122896s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:02.063709030 (7h23m15.650765241s) [upload=7574]: Done uploading segments... Aug 7 23:50:02.063723547 (7h23m15.650779768s) [upload=7574]: Upload finished. err: <nil> Aug 7 23:50:41.160166556 (7h23m54.747222777s) [upload=7588]: Starting upload Aug 7 23:50:41.160220555 (7h23m54.747276777s) [upload=7588]: Done starting upload Aug 7 23:50:41.160264659 (7h23m54.74732088s) [upload=7588]: Uploading segments... Aug 7 23:50:41.160295956 (7h23m54.747352177s) [upload=7588]: Got next segment. Inline: true Aug 7 23:50:41.160299753 (7h23m54.747355964s) [upload=7588]: Waiting for error group managing segments... Aug 7 23:50:41.160331633 (7h23m54.747387844s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:41.160335861 (7h23m54.747392072s) [upload=7588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:41.160338396 (7h23m54.747394617s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:41.222859735 (7h23m54.809915976s) [upload=7588]: Done uploading segments... Aug 7 23:50:41.222879913 (7h23m54.809936154s) [upload=7588]: Upload finished. err: <nil> Aug 7 23:51:17.131690986 (7h24m30.718747197s) [upload=7599]: Starting upload Aug 7 23:51:17.131738926 (7h24m30.718795147s) [upload=7599]: Done starting upload Aug 7 23:51:17.131792947 (7h24m30.718849178s) [upload=7599]: Uploading segments... Aug 7 23:51:17.131835896 (7h24m30.718892117s) [upload=7599]: Got next segment. Inline: true Aug 7 23:51:17.131842027 (7h24m30.718898239s) [upload=7599]: Waiting for error group managing segments... Aug 7 23:51:17.131883385 (7h24m30.718939627s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:17.131889146 (7h24m30.718945388s) [upload=7599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:17.132006205 (7h24m30.719062435s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:38.787433326 (7h24m52.374489567s) [upload=7599]: Done uploading segments... Aug 7 23:51:38.787452212 (7h24m52.374508453s) [upload=7599]: Upload finished. err: <nil> Aug 7 23:52:00.256204647 (7h25m13.843260858s) [upload=7610]: Starting upload Aug 7 23:52:00.256263547 (7h25m13.843319758s) [upload=7610]: Done starting upload Aug 7 23:52:00.256316195 (7h25m13.843372406s) [upload=7610]: Uploading segments... Aug 7 23:52:00.256345671 (7h25m13.843401881s) [upload=7610]: Got next segment. Inline: true Aug 7 23:52:00.256349758 (7h25m13.843405969s) [upload=7610]: Waiting for error group managing segments... Aug 7 23:52:00.256399992 (7h25m13.843456203s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:00.256404780 (7h25m13.843460991s) [upload=7610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:00.256407806 (7h25m13.843464017s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:30.237600164 (7h25m43.824656416s) [upload=7610]: Done uploading segments... Aug 7 23:52:30.237630862 (7h25m43.824687113s) [upload=7610]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:52:53.146251246 (7h26m6.733307456s) [upload=7626]: Starting upload Aug 7 23:52:53.146307339 (7h26m6.733363561s) [upload=7626]: Done starting upload Aug 7 23:52:53.146350962 (7h26m6.733407173s) [upload=7626]: Uploading segments... Aug 7 23:52:53.146411344 (7h26m6.733467565s) [upload=7626]: Got next segment. Inline: true Aug 7 23:52:53.146415271 (7h26m6.733471493s) [upload=7626]: Waiting for error group managing segments... Aug 7 23:52:53.146449966 (7h26m6.733506177s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:53.146454174 (7h26m6.733510395s) [upload=7626]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:53.146456709 (7h26m6.73351293s) [upload=7626]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:23.132536521 (7h26m36.719592742s) [upload=7626]: Done uploading segments... Aug 7 23:53:23.132560907 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.000969967s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 173.725377ms FAIL (2025-08-07 23:53:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 173.749432ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........XXX.X.X.........................................X] duration: 1m0.000829858s FAIL (2025-08-07 23:53:11 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== QBRGXTTEOU4SJY2SWUTFQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.346810570 (7h24m30.933866781s) [noise=true , address=104.36.82.42:4801 , piece_id=ILNDV4CZCPEST6R4DUTSHQ5OJS2N4BLPRLDYIYNBTZKRR4RSCOFA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.346916297 (7h24m30.933972519s) [noise=true , address=104.36.82.59:4327 , piece_id=5AM7TVDV5ORD5WPHKEHOM652JETCJ6VH3JOJ6HPJTIOTABTYVELA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347112754 (7h24m30.934168975s) [noise=true , address=104.36.82.44:4904 , piece_id=TFXFZ53DGPOVVXTTZMRMTJR53DGXNHWUF43I7OCC4YNCRW4QUCKA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347141428 (7h24m30.934197638s) [noise=true , address=79.127.201.215:4343 , piece_id=6IVWBNSJNI4ZAEIWFTRYUHIHZ35YTTZF3FTEF7G76YEBUP2NBJFA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347399839 (7h24m30.934456051s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347417763 (7h24m30.934473974s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347625520 (7h24m30.934681731s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347755754 (7h24m30.934811985s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.347968761 (7h24m30.935024982s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348133560 (7h24m30.935189781s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348193100 (7h24m30.935249311s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348536501 (7h24m30.935592722s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348656385 (7h24m30.935712596s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348854374 (7h24m30.935910585s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.348919236 (7h24m30.935975457s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Uploading piece... Aug 7 23:51:17.430583697 (7h24m31.017639918s) [upload=7600]: Next returned nil segment Aug 7 23:51:17.430596350 (7h24m31.017652571s) [upload=7600]: Waiting for error group managing segments... Aug 7 23:51:17.450496095 (7h24m31.037552306s) [noise=true , address=104.36.82.39:4729 , piece_id=B7IGTQUWZAQNQDUYOUAB3WGYK5QHKXXJKRDGKMEGDK6KS4ZFC5GA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.457273115 (7h24m31.044329336s) [noise=true , address=45.91.116.11:4121 , piece_id=UI4HOUG4KLWC3LFGDVZ3C3IUMB2DGJP2QOTIOLWS5C2HUAMSATMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.458658320 (7h24m31.045714532s) [noise=true , address=104.36.82.36:4649 , piece_id=NV2V22M4HHIEHH4RENXMFHFYXHQF7LPJRPPD7ZBKHPSZFKI2BVTA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.464324767 (7h24m31.051380978s) [noise=true , address=104.36.82.62:4438 , piece_id=3ND3RVOIIEJMN5GH7KZRHPYVNIH5KB62OU6KEV5DVOLCAZI4BQLA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.469258046 (7h24m31.056314256s) [noise=true , address=104.36.82.4:4806 , piece_id=X42UHVJVQMP46Y7QDE377JH6FNETULCU4JBDSGTVJDRSUE5LAVOA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.470547041 (7h24m31.057603252s) [noise=true , address=45.91.116.10:4098 , piece_id=5ITNSAGAAD2QAEAZGQSRYU2GSIS54FPDHU6HETL6QVR57KUDYICQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.482895480 (7h24m31.069951691s) [noise=true , address=104.36.82.59:4327 , piece_id=5AM7TVDV5ORD5WPHKEHOM652JETCJ6VH3JOJ6HPJTIOTABTYVELA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.503848269 (7h24m31.090904481s) [noise=true , address=104.36.82.61:4406 , piece_id=N63ZWSHAILE55ZOUL5H6RYDUTB5F2F3EWMDT52RPJ5F46MLKRDMA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.589438619 (7h24m31.17649484s) [noise=true , address=79.127.201.215:4343 , piece_id=6IVWBNSJNI4ZAEIWFTRYUHIHZ35YTTZF3FTEF7G76YEBUP2NBJFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.633863581 (7h24m31.220919802s) [noise=true , address=104.36.82.42:4801 , piece_id=ILNDV4CZCPEST6R4DUTSHQ5OJS2N4BLPRLDYIYNBTZKRR4RSCOFA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.636493760 (7h24m31.22354997s) [noise=true , address=104.36.82.44:4904 , piece_id=TFXFZ53DGPOVVXTTZMRMTJR53DGXNHWUF43I7OCC4YNCRW4QUCKA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.637613389 (7h24m31.224669611s) [noise=true , address=104.36.82.23:4475 , piece_id=LZMIGPR677ZHFYTXZJAMLW3GC3XLON7LGPRRBWZWVEXG4GGLEEEA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.654845844 (7h24m31.241902065s) [noise=true , address=104.36.82.22:4524 , piece_id=66YIUJKSDV52T5NMMFHYCZH7L34UKH3RLIS5WZ3VWRBL3KEE7UWQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853873712 (7h24m31.440929933s) [noise=true , address=104.36.82.12:4178 , piece_id=NQWZGYOLKX6SHV3RXIXVGFDAXFP4VMFMP5FJKQPZ63YZ2NK3RJDQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: <nil> Aug 7 23:51:17.853888850 (7h24m31.440945061s) [seg_pos={0 0}, upload=7600]: Segment reached optimal threshold of 14 pieces. Aug 7 23:51:17.853987013 (7h24m31.441043234s) [noise=true , address=104.36.82.45:4921 , piece_id=GQXCWGC6EHWZONLT426WVNPLASR7SECUANPQGGZNMASCRRD4X4SQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bBRKXiuSN2cw6KrdP8q4Dsf4MiFCUSgMEGjqG2dKL9RLTQra1): protocol: expected piece hash; context canceled; EOF Aug 7 23:51:17.854295209 (7h24m31.44135142s) [noise=true , address=104.36.82.46:4499 , piece_id=6OBTEQJKT4AJRGUT27EYWNLG4E2TKBCQBRGXTTEOU4SJY2SWUTFQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JR5bVTizvQCY5utaGUDgLHvCJfrbbpixmKNrq5vjiRB8VaSoc): protocol: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:63506->104.36.82.46:4499: use of closed network connection Aug 7 23:51:17.854338279 (7h24m31.44139449s) [noise=true , address=104.36.82.35:4611 , piece_id=FE47XNPAFZAELTDVIO6C4URDMXRWEXCM37BFM4LANQQ6QDSU7HAQ, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zkqEMSKx7zSHPFNZ51b5TLzL2cFQL8VyxAG14Lg1pT8zAu3tc): protocol: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:46266->104.36.82.35:4611: use of closed network connection Aug 7 23:51:17.854399554 (7h24m31.441455765s) [noise=true , address=104.36.82.15:4288 , piece_id=KWW7E4ZMO42C3YEAH3WBCEYSW6TTNDX7TQ4MEITLSC2RR2CE4VDA, seg_pos={0 0}, upload=7600]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gPjPGuueZnizLT2QvJN9y7riwKiW5TWMkQtmGGPP7VZvkNoB7): protocol: read tcp 5.161.89.252:35428->104.36.82.15:4288: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:51:17.854419941 (7h24m31.441476162s) [seg_pos={0 0}, upload=7600]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:51:17.854472530 (7h24m31.441528751s) [upload=7600]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:51:17.854476738 (7h24m31.441532959s) [upload=7600]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:17.900817215 (7h24m31.487873465s) [upload=7600]: Done uploading segments... Aug 7 23:51:17.900834827 (7h24m31.487891079s) [upload=7600]: Upload finished. err: <nil> Aug 7 23:52:11.382772324 (7h25m24.969828534s) [upload=7616]: Starting upload Aug 7 23:52:11.382828459 (7h25m24.96988468s) [upload=7616]: Done starting upload Aug 7 23:52:11.382871529 (7h25m24.96992774s) [upload=7616]: Uploading segments... Aug 7 23:52:11.383053639 (7h25m24.97010986s) [upload=7616]: Got next segment. Inline: false Aug 7 23:52:11.383080680 (7h25m24.970136891s) [upload=7616]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:11.383084406 (7h25m24.970140617s) [upload=7616]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:53:11.371970391 (7h26m24.959026631s) [upload=7616]: Next segment err: context deadline exceeded Aug 7 23:53:11.372056972 (7h26m24.959113223s) [upload=7616]: Done uploading segments... Aug 7 23:53:11.372090856 (7h26m24.959147096s) [upload=7616]: Upload finished. err: context deadline exceeded Duration: 1m0.000851269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 857.299601ms FAIL (2025-08-07 23:53:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 857.322764ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [......XXXX.X..X..X......................................XXX.] duration: 307.684452ms FAIL (2025-08-07 23:53:03 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 307.713115ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.......XXXXX..X............................................X] duration: 1m0.000570826s FAIL (2025-08-07 23:53:00 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000586204s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [........X.XXXXX..X..........................................] duration: 189.055884ms FAIL (2025-08-07 23:52:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 189.115705ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.........X.XXXXX..X.........................................] duration: 215.79334ms FAIL (2025-08-07 23:52:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 215.814041ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....XXXXXX.X.X...........................................X.] duration: 1.652674115s FAIL (2025-08-07 23:52:33 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.652692058s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [......XXXXXX.X.X...........................................X] duration: 30.001304991s FAIL (2025-08-07 23:52:31 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001322794s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 9 out of 60 is failed (threshold=3) [......XXXXX.XX............................................XX] duration: 30.000516658s FAIL (2025-08-07 23:52:30 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 0330 (7h17m25.727496541s) [upload=7481]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:12.194949697 (7h17m25.782005918s) [upload=7481]: Done uploading segments... Aug 7 23:44:12.194964323 (7h17m25.782020544s) [upload=7481]: Upload finished. err: <nil> Aug 7 23:44:35.832126527 (7h17m49.419182778s) [upload=7488]: Starting upload Aug 7 23:44:35.832228848 (7h17m49.41928514s) [upload=7488]: Done starting upload Aug 7 23:44:35.832360132 (7h17m49.419416384s) [upload=7488]: Uploading segments... Aug 7 23:44:35.832419373 (7h17m49.419475615s) [upload=7488]: Got next segment. Inline: true Aug 7 23:44:35.832428821 (7h17m49.419485062s) [upload=7488]: Waiting for error group managing segments... Aug 7 23:44:35.832486639 (7h17m49.419542889s) [upload=7488]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:44:35.832495696 (7h17m49.419551936s) [upload=7488]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:44:35.832501827 (7h17m49.419558068s) [upload=7488]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:44:35.894656303 (7h17m49.481712545s) [upload=7488]: Done uploading segments... Aug 7 23:44:35.894676211 (7h17m49.481732452s) [upload=7488]: Upload finished. err: <nil> Aug 7 23:45:42.609270547 (7h18m56.196326759s) [upload=7507]: Starting upload Aug 7 23:45:42.609334396 (7h18m56.196390617s) [upload=7507]: Done starting upload Aug 7 23:45:42.609380774 (7h18m56.196436984s) [upload=7507]: Uploading segments... Aug 7 23:45:42.609412683 (7h18m56.196468914s) [upload=7507]: Got next segment. Inline: true Aug 7 23:45:42.609418174 (7h18m56.196474385s) [upload=7507]: Waiting for error group managing segments... Aug 7 23:45:42.609455492 (7h18m56.196511713s) [upload=7507]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:45:42.609459600 (7h18m56.196515811s) [upload=7507]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:45:42.609462095 (7h18m56.196518316s) [upload=7507]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:45:42.664256383 (7h18m56.251312604s) [upload=7507]: Done uploading segments... Aug 7 23:45:42.664275208 (7h18m56.251331429s) [upload=7507]: Upload finished. err: <nil> Aug 7 23:46:08.906811448 (7h19m22.493867688s) [upload=7514]: Starting upload Aug 7 23:46:08.906885386 (7h19m22.493941627s) [upload=7514]: Done starting upload Aug 7 23:46:08.906964935 (7h19m22.494021175s) [upload=7514]: Uploading segments... Aug 7 23:46:08.906997987 (7h19m22.494054228s) [upload=7514]: Got next segment. Inline: true Aug 7 23:46:08.907003197 (7h19m22.494059437s) [upload=7514]: Waiting for error group managing segments... Aug 7 23:46:08.907046577 (7h19m22.494102807s) [upload=7514]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:46:08.907051767 (7h19m22.494108007s) [upload=7514]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:46:08.907055173 (7h19m22.494111414s) [upload=7514]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:46:08.965075092 (7h19m22.552131313s) [upload=7514]: Done uploading segments... Aug 7 23:46:08.965095970 (7h19m22.552152191s) [upload=7514]: Upload finished. err: <nil> Aug 7 23:47:00.017908282 (7h20m13.604964493s) [upload=7527]: Starting upload Aug 7 23:47:00.017955210 (7h20m13.605011411s) [upload=7527]: Done starting upload Aug 7 23:47:00.017998241 (7h20m13.605054452s) [upload=7527]: Uploading segments... Aug 7 23:47:00.018026292 (7h20m13.605082503s) [upload=7527]: Got next segment. Inline: true Aug 7 23:47:00.018029438 (7h20m13.60508565s) [upload=7527]: Waiting for error group managing segments... Aug 7 23:47:00.018058453 (7h20m13.605114664s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:47:00.018062390 (7h20m13.605118601s) [upload=7527]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:47:00.018064304 (7h20m13.605120515s) [upload=7527]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:00.075526210 (7h20m13.662582431s) [upload=7527]: Done uploading segments... Aug 7 23:47:00.075540917 (7h20m13.662597138s) [upload=7527]: Upload finished. err: <nil> Aug 7 23:48:09.882765252 (7h21m23.469821463s) [upload=7545]: Starting upload Aug 7 23:48:09.882860198 (7h21m23.46991642s) [upload=7545]: Done starting upload Aug 7 23:48:09.882935729 (7h21m23.469991941s) [upload=7545]: Uploading segments... Aug 7 23:48:09.882965635 (7h21m23.470021856s) [upload=7545]: Got next segment. Inline: true Aug 7 23:48:09.882969723 (7h21m23.470025944s) [upload=7545]: Waiting for error group managing segments... Aug 7 23:48:09.883002865 (7h21m23.470059086s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:09.883006481 (7h21m23.470062702s) [upload=7545]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:09.883009045 (7h21m23.470065266s) [upload=7545]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:09.951272229 (7h21m23.538328451s) [upload=7545]: Done uploading segments... Aug 7 23:48:09.951286926 (7h21m23.538343148s) [upload=7545]: Upload finished. err: <nil> Aug 7 23:48:58.772502770 (7h22m12.359559011s) [upload=7557]: Starting upload Aug 7 23:48:58.772574805 (7h22m12.359631046s) [upload=7557]: Done starting upload Aug 7 23:48:58.772645527 (7h22m12.359701767s) [upload=7557]: Uploading segments... Aug 7 23:48:58.772695300 (7h22m12.359751541s) [upload=7557]: Got next segment. Inline: true Aug 7 23:48:58.772701031 (7h22m12.359757271s) [upload=7557]: Waiting for error group managing segments... Aug 7 23:48:58.772770249 (7h22m12.359826501s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:58.772780228 (7h22m12.359836469s) [upload=7557]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:58.772787010 (7h22m12.359843252s) [upload=7557]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:58.840436652 (7h22m12.427492893s) [upload=7557]: Done uploading segments... Aug 7 23:48:58.840462661 (7h22m12.427518891s) [upload=7557]: Upload finished. err: <nil> Aug 7 23:50:01.971851413 (7h23m15.558907624s) [upload=7574]: Starting upload Aug 7 23:50:01.971890696 (7h23m15.558946907s) [upload=7574]: Done starting upload Aug 7 23:50:01.971977749 (7h23m15.559033961s) [upload=7574]: Uploading segments... Aug 7 23:50:01.972022943 (7h23m15.559079154s) [upload=7574]: Got next segment. Inline: true Aug 7 23:50:01.972026850 (7h23m15.559083062s) [upload=7574]: Waiting for error group managing segments... Aug 7 23:50:01.972060975 (7h23m15.559117185s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:01.972064511 (7h23m15.559120712s) [upload=7574]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:01.972066685 (7h23m15.559122896s) [upload=7574]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:02.063709030 (7h23m15.650765241s) [upload=7574]: Done uploading segments... Aug 7 23:50:02.063723547 (7h23m15.650779768s) [upload=7574]: Upload finished. err: <nil> Aug 7 23:50:41.160166556 (7h23m54.747222777s) [upload=7588]: Starting upload Aug 7 23:50:41.160220555 (7h23m54.747276777s) [upload=7588]: Done starting upload Aug 7 23:50:41.160264659 (7h23m54.74732088s) [upload=7588]: Uploading segments... Aug 7 23:50:41.160295956 (7h23m54.747352177s) [upload=7588]: Got next segment. Inline: true Aug 7 23:50:41.160299753 (7h23m54.747355964s) [upload=7588]: Waiting for error group managing segments... Aug 7 23:50:41.160331633 (7h23m54.747387844s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:41.160335861 (7h23m54.747392072s) [upload=7588]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:41.160338396 (7h23m54.747394617s) [upload=7588]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:41.222859735 (7h23m54.809915976s) [upload=7588]: Done uploading segments... Aug 7 23:50:41.222879913 (7h23m54.809936154s) [upload=7588]: Upload finished. err: <nil> Aug 7 23:51:17.131690986 (7h24m30.718747197s) [upload=7599]: Starting upload Aug 7 23:51:17.131738926 (7h24m30.718795147s) [upload=7599]: Done starting upload Aug 7 23:51:17.131792947 (7h24m30.718849178s) [upload=7599]: Uploading segments... Aug 7 23:51:17.131835896 (7h24m30.718892117s) [upload=7599]: Got next segment. Inline: true Aug 7 23:51:17.131842027 (7h24m30.718898239s) [upload=7599]: Waiting for error group managing segments... Aug 7 23:51:17.131883385 (7h24m30.718939627s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:17.131889146 (7h24m30.718945388s) [upload=7599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:17.132006205 (7h24m30.719062435s) [upload=7599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:38.787433326 (7h24m52.374489567s) [upload=7599]: Done uploading segments... Aug 7 23:51:38.787452212 (7h24m52.374508453s) [upload=7599]: Upload finished. err: <nil> Aug 7 23:52:00.256204647 (7h25m13.843260858s) [upload=7610]: Starting upload Aug 7 23:52:00.256263547 (7h25m13.843319758s) [upload=7610]: Done starting upload Aug 7 23:52:00.256316195 (7h25m13.843372406s) [upload=7610]: Uploading segments... Aug 7 23:52:00.256345671 (7h25m13.843401881s) [upload=7610]: Got next segment. Inline: true Aug 7 23:52:00.256349758 (7h25m13.843405969s) [upload=7610]: Waiting for error group managing segments... Aug 7 23:52:00.256399992 (7h25m13.843456203s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:00.256404780 (7h25m13.843460991s) [upload=7610]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:00.256407806 (7h25m13.843464017s) [upload=7610]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:30.237600164 (7h25m43.824656416s) [upload=7610]: Done uploading segments... Aug 7 23:52:30.237630862 (7h25m43.824687113s) [upload=7610]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.000538539s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.......XXXX.X..X..X......................................XXX] duration: 30.001485608s FAIL (2025-08-07 23:52:30 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001512159s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 823.057005ms FAIL (2025-08-07 23:52:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 823.076712ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X.XX.X....X.X..........................................] duration: 450.885344ms FAIL (2025-08-07 23:52:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 450.971134ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 202.529242ms FAIL (2025-08-07 23:52:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 202.579748ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.....XXXXXX.X...............................................] duration: 342.649335ms FAIL (2025-08-07 23:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== segments... Aug 7 23:10:36.018385587 (6h43m49.605441827s) [upload=6905]: Upload finished. err: <nil> Aug 7 23:10:57.038963659 (6h44m10.62601988s) [upload=6913]: Starting upload Aug 7 23:10:57.039017149 (6h44m10.62607337s) [upload=6913]: Done starting upload Aug 7 23:10:57.039065159 (6h44m10.62612137s) [upload=6913]: Uploading segments... Aug 7 23:10:57.039093831 (6h44m10.626150042s) [upload=6913]: Got next segment. Inline: true Aug 7 23:10:57.039097308 (6h44m10.626153519s) [upload=6913]: Waiting for error group managing segments... Aug 7 23:10:57.039142663 (6h44m10.626198874s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:57.039146320 (6h44m10.626202531s) [upload=6913]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:57.039148564 (6h44m10.626204785s) [upload=6913]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:57.092133855 (6h44m10.679190086s) [upload=6913]: Done uploading segments... Aug 7 23:10:57.092152090 (6h44m10.67920833s) [upload=6913]: Upload finished. err: <nil> Aug 7 23:11:47.690012005 (6h45m1.277068256s) [upload=6926]: Starting upload Aug 7 23:11:47.690127641 (6h45m1.277183882s) [upload=6926]: Done starting upload Aug 7 23:11:47.690277249 (6h45m1.27733349s) [upload=6926]: Uploading segments... Aug 7 23:11:47.690316773 (6h45m1.277373025s) [upload=6926]: Got next segment. Inline: true Aug 7 23:11:47.690360005 (6h45m1.277416245s) [upload=6926]: Waiting for error group managing segments... Aug 7 23:11:47.690405779 (6h45m1.277462031s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:47.690411880 (6h45m1.277468122s) [upload=6926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:47.690415508 (6h45m1.277471748s) [upload=6926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:47.740982296 (6h45m1.328038537s) [upload=6926]: Done uploading segments... Aug 7 23:11:47.740995380 (6h45m1.328051622s) [upload=6926]: Upload finished. err: <nil> Aug 7 23:12:15.388374517 (6h45m28.975430749s) [upload=6935]: Starting upload Aug 7 23:12:15.388464206 (6h45m28.975520437s) [upload=6935]: Done starting upload Aug 7 23:12:15.388550818 (6h45m28.975607039s) [upload=6935]: Uploading segments... Aug 7 23:12:15.388651916 (6h45m28.975708127s) [upload=6935]: Got next segment. Inline: true Aug 7 23:12:15.388660222 (6h45m28.975716443s) [upload=6935]: Waiting for error group managing segments... Aug 7 23:12:15.388716747 (6h45m28.975772968s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:15.388723729 (6h45m28.97577995s) [upload=6935]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:15.388727838 (6h45m28.975784059s) [upload=6935]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:15.375897715 (6h46m28.962953947s) [upload=6935]: Done uploading segments... Aug 7 23:13:15.375942599 (6h46m28.962998841s) [upload=6935]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:13:15.452799710 (6h46m29.039855931s) [upload=6951]: Starting upload Aug 7 23:13:15.452856376 (6h46m29.039912597s) [upload=6951]: Done starting upload Aug 7 23:13:15.452901079 (6h46m29.0399573s) [upload=6951]: Uploading segments... Aug 7 23:13:15.452938739 (6h46m29.03999495s) [upload=6951]: Got next segment. Inline: true Aug 7 23:13:15.452943217 (6h46m29.039999439s) [upload=6951]: Waiting for error group managing segments... Aug 7 23:13:15.452974977 (6h46m29.040031188s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:15.452979305 (6h46m29.040035526s) [upload=6951]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:15.452982110 (6h46m29.040038331s) [upload=6951]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:15.442587501 (6h47m29.029643742s) [upload=6951]: Done uploading segments... Aug 7 23:14:15.442608670 (6h47m29.029664922s) [upload=6951]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:14:15.516163727 (6h47m29.103219978s) [upload=6968]: Starting upload Aug 7 23:14:15.516314939 (6h47m29.10337124s) [upload=6968]: Done starting upload Aug 7 23:14:15.516411479 (6h47m29.103467721s) [upload=6968]: Uploading segments... Aug 7 23:14:15.516477332 (6h47m29.103533573s) [upload=6968]: Got next segment. Inline: true Aug 7 23:14:15.516487672 (6h47m29.103543913s) [upload=6968]: Waiting for error group managing segments... Aug 7 23:14:15.516556882 (6h47m29.103613162s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:15.516567169 (6h47m29.103623411s) [upload=6968]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:15.516573491 (6h47m29.103629733s) [upload=6968]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:15.501845849 (6h48m29.08890207s) [upload=6968]: Done uploading segments... Aug 7 23:15:15.501865025 (6h48m29.088921247s) [upload=6968]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:15:15.591828711 (6h48m29.178884922s) [upload=6986]: Starting upload Aug 7 23:15:15.591883403 (6h48m29.178939613s) [upload=6986]: Done starting upload Aug 7 23:15:15.591959133 (6h48m29.179015344s) [upload=6986]: Uploading segments... Aug 7 23:15:15.592001994 (6h48m29.179058205s) [upload=6986]: Got next segment. Inline: true Aug 7 23:15:15.592022743 (6h48m29.179078964s) [upload=6986]: Waiting for error group managing segments... Aug 7 23:15:15.592060944 (6h48m29.179117156s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:15.592065653 (6h48m29.179121874s) [upload=6986]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:15.592067957 (6h48m29.179124178s) [upload=6986]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:15.578451341 (6h49m29.165507562s) [upload=6986]: Done uploading segments... Aug 7 23:16:15.578469606 (6h49m29.165525816s) [upload=6986]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:16:23.278937820 (6h49m36.865994061s) [upload=7004]: Starting upload Aug 7 23:16:23.279020784 (6h49m36.866077025s) [upload=7004]: Done starting upload Aug 7 23:16:23.279084963 (6h49m36.866141205s) [upload=7004]: Uploading segments... Aug 7 23:16:23.279128285 (6h49m36.866184526s) [upload=7004]: Got next segment. Inline: true Aug 7 23:16:23.279135057 (6h49m36.866191299s) [upload=7004]: Waiting for error group managing segments... Aug 7 23:16:23.279179039 (6h49m36.86623528s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:23.279184509 (6h49m36.866240751s) [upload=7004]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:23.279188327 (6h49m36.866244558s) [upload=7004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:23.264696428 (6h50m36.851752649s) [upload=7004]: Done uploading segments... Aug 7 23:17:23.264712168 (6h50m36.851768379s) [upload=7004]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:17:23.354485739 (6h50m36.94154196s) [upload=7020]: Starting upload Aug 7 23:17:23.354537225 (6h50m36.941593446s) [upload=7020]: Done starting upload Aug 7 23:17:23.354582019 (6h50m36.94163824s) [upload=7020]: Uploading segments... Aug 7 23:17:23.354609931 (6h50m36.941666142s) [upload=7020]: Got next segment. Inline: true Aug 7 23:17:23.354613586 (6h50m36.941669807s) [upload=7020]: Waiting for error group managing segments... Aug 7 23:17:23.354647150 (6h50m36.941703371s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:23.354650656 (6h50m36.941706877s) [upload=7020]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:17:23.354652961 (6h50m36.941709172s) [upload=7020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:23.336455483 (6h51m36.923511723s) [upload=7020]: Done uploading segments... Aug 7 23:18:23.336475711 (6h51m36.923531952s) [upload=7020]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:18:34.755118608 (6h51m48.342174818s) [upload=7039]: Starting upload Aug 7 23:18:34.755161578 (6h51m48.342217799s) [upload=7039]: Done starting upload Aug 7 23:18:34.755200670 (6h51m48.342256881s) [upload=7039]: Uploading segments... Aug 7 23:18:34.755239954 (6h51m48.342296176s) [upload=7039]: Got next segment. Inline: true Aug 7 23:18:34.755243802 (6h51m48.342300023s) [upload=7039]: Waiting for error group managing segments... Aug 7 23:18:34.755274309 (6h51m48.34233052s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:18:34.755277996 (6h51m48.342334207s) [upload=7039]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:18:34.755280250 (6h51m48.342336471s) [upload=7039]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:18:34.875891722 (6h51m48.462947943s) [upload=7039]: Done uploading segments... Aug 7 23:18:34.875937538 (6h51m48.462993759s) [upload=7039]: Upload finished. err: <nil> Aug 7 23:19:14.105729045 (6h52m27.692785266s) [upload=7050]: Starting upload Aug 7 23:19:14.105780641 (6h52m27.692836852s) [upload=7050]: Done starting upload Aug 7 23:19:14.105815075 (6h52m27.692871286s) [upload=7050]: Uploading segments... Aug 7 23:19:14.105851563 (6h52m27.692907773s) [upload=7050]: Got next segment. Inline: true Aug 7 23:19:14.105856422 (6h52m27.692912643s) [upload=7050]: Waiting for error group managing segments... Aug 7 23:19:14.105888662 (6h52m27.692944883s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:19:14.105892429 (6h52m27.69294864s) [upload=7050]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:19:14.105894724 (6h52m27.692950934s) [upload=7050]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:20:14.092509460 (6h53m27.679565681s) [upload=7050]: Done uploading segments... Aug 7 23:20:14.092523326 (6h53m27.679579547s) [upload=7050]: Upload finished. err: metaclient: context deadline exceeded Duration: 342.672808ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X....XXX.X......................................] duration: 796.682846ms FAIL (2025-08-07 23:52:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ite tcp 5.161.89.252:59232->104.36.82.13:4187: use of closed network connection Aug 7 23:17:27.468792451 (6h50m41.055848663s) [noise=true , address=104.36.82.15:4257 , piece_id=YSUUNUNXXR4XZZRQBZVTYSP7WZNQVDHHS7KVVRKIAY5AQI3EMGCQ, seg_pos={0 0}, upload=7021]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: read tcp 5.161.89.252:16228->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:17:27.468805356 (6h50m41.055861567s) [seg_pos={0 0}, upload=7021]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:17:27.468898489 (6h50m41.055954701s) [upload=7021]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:17:27.468902838 (6h50m41.055959049s) [upload=7021]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:17:27.512149772 (6h50m41.099205993s) [upload=7021]: Done uploading segments... Aug 7 23:17:27.512163628 (6h50m41.099219849s) [upload=7021]: Upload finished. err: <nil> Aug 7 23:17:51.618188257 (6h51m5.205244499s) [upload=7028]: Starting upload Aug 7 23:17:51.618315174 (6h51m5.205371416s) [upload=7028]: Done starting upload Aug 7 23:17:51.618432584 (6h51m5.205488824s) [upload=7028]: Uploading segments... Aug 7 23:17:51.618706434 (6h51m5.205762675s) [upload=7028]: Got next segment. Inline: false Aug 7 23:17:51.618753392 (6h51m5.205809634s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:17:51.618759304 (6h51m5.205815554s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:18:02.217185080 (6h51m15.804241291s) [seg_pos={0 0}, upload=7028]: Begin upload segment... Aug 7 23:18:02.217213693 (6h51m15.804269904s) [seg_pos={0 0}, upload=7028]: Done begin upload segment. Aug 7 23:18:02.217258727 (6h51m15.804314948s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217278985 (6h51m15.804335206s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217653263 (6h51m15.804709484s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217754012 (6h51m15.804810233s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.217977880 (6h51m15.805034101s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218105467 (6h51m15.805161678s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218314438 (6h51m15.805370659s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218477351 (6h51m15.805533563s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218633864 (6h51m15.805690076s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218759268 (6h51m15.80581548s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.218945636 (6h51m15.806001857s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219088874 (6h51m15.806145095s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219280120 (6h51m15.806336341s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219439768 (6h51m15.806495989s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219602472 (6h51m15.806658693s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.219943268 (6h51m15.806999489s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Uploading piece... Aug 7 23:18:02.309104326 (6h51m15.896160547s) [upload=7028]: Next returned nil segment Aug 7 23:18:02.309119103 (6h51m15.896175324s) [upload=7028]: Waiting for error group managing segments... Aug 7 23:18:02.326561371 (6h51m15.913617592s) [noise=true , address=45.91.116.14:4190 , piece_id=VN7ZRT4NTNUNQNKN2OCNILBFMN2QNGZOOY6B6EFDPET3TIB42A7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.327084597 (6h51m15.914140818s) [noise=true , address=45.91.116.7:4009 , piece_id=VMY27LD3MBT4O65CXJJEYBIP7QVB543GSJOTQLYBELK4G7DOBQRA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329365706 (6h51m15.916421916s) [noise=true , address=45.91.116.6:4974 , piece_id=MQWUSYBCOBJPCBLKUWG5R2YGX2UXHJAYTFXLU5G45I7ADEEJOSIA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.329710399 (6h51m15.916766611s) [noise=true , address=77.90.31.14:4662 , piece_id=FWDHERWXZUN4WAURKVGP6UC7PC57XBLLIDHJFAPEP6YUUHK7XSGA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330040726 (6h51m15.917096936s) [noise=true , address=77.90.31.17:4772 , piece_id=A7XUBHQKTJRVAMBQAO24KOU4GFGENHMFY6DBETNJDZ7J5YR7VO6Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330501876 (6h51m15.917558097s) [noise=true , address=77.90.31.18:4806 , piece_id=2L4OYP2PJ6LBOBZUUI52QNVFWAOMU7E25SJ7XB6C2CQCAREMFVQA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.330860886 (6h51m15.917917107s) [noise=true , address=77.90.31.16:4735 , piece_id=QBT3GKFTCJBCONV4W4E3V74IQBXZO2R4M3SQBJBGRPEXMN4L4V2A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.331216931 (6h51m15.918273152s) [noise=true , address=77.90.31.12:4592 , piece_id=X4RIDUL5RUENCF4ZVRNIBWIUAHGQYW6XLBZWDMOUYGEIWQL5ZH7A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332064072 (6h51m15.919120293s) [noise=true , address=77.90.31.21:4877 , piece_id=C7HY6GFRIWXX43OOGKBM3EXJVIMPUQP36KHHSVSGOVFJYBBX4IBA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332119265 (6h51m15.919175476s) [noise=true , address=77.90.31.11:4553 , piece_id=SJUK6H5G242A3QOXRARMJGQP6IGRQ2WJOUHBK24BMT2M3DYKGKAQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.332931742 (6h51m15.919987963s) [noise=true , address=104.36.82.3:4767 , piece_id=D6EYZVFHGH7J3ABUMJBQPTJUDNA6GPMONDLQY7WL6NTVLYEEZM4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.340533561 (6h51m15.927589782s) [noise=true , address=45.91.116.10:4082 , piece_id=GCFQIVXLPEIWQXT4A5Y6Z4N2E54HKSLGB2QDXR62SYQ4AISIXIMA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345624544 (6h51m15.932680765s) [noise=true , address=104.36.82.14:4222 , piece_id=BJCIBUJTQDIS64F2XTL6XYNMICZJUXBP2NUA3UXOCORROZWONL4Q, seg_pos={0 0}, upload=7028]: Done uploading piece. err: <nil> Aug 7 23:18:02.345641816 (6h51m15.932698037s) [seg_pos={0 0}, upload=7028]: Segment reached optimal threshold of 13 pieces. Aug 7 23:18:02.345736042 (6h51m15.932792253s) [noise=true , address=104.36.82.11:4115 , piece_id=MIW4TCCVX7O3KTEBU2RQRHD4GKPBYQOBTDKXGLH3B3NYCKTBDQ3A, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345762481 (6h51m15.932818693s) [noise=true , address=77.90.31.19:4842 , piece_id=2K54EHW7ZB22YIUFSMKZBKD66T4NLCPLSAPR344HPTF7WPX537JA, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Aug 7 23:18:02.345881584 (6h51m15.932937805s) [noise=true , address=104.36.82.9:4006 , piece_id=3F6A7S5CWE2TOBI6R77DHSEIWGL3HBLYRWSJKC7PGTN747XL77NQ, seg_pos={0 0}, upload=7028]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: context canceled; context canceled Aug 7 23:18:02.345898606 (6h51m15.932954827s) [seg_pos={0 0}, upload=7028]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:18:02.345971481 (6h51m15.933027702s) [upload=7028]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:18:02.345975489 (6h51m15.93303171s) [upload=7028]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:22:03.557058290 (6h55m17.144114511s) [upload=7028]: Done uploading segments... Aug 7 23:22:03.557078478 (6h55m17.144134699s) [upload=7028]: Upload finished. err: metaclient: context deadline exceeded Duration: 796.704105ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 9 out of 60 is failed (threshold=3) [........XXXX.X..X..X......................................XX] duration: 30.001308857s FAIL (2025-08-07 23:52:00 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001327423s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)