Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [X....................................................X......] duration: 216.908356ms FAIL (2025-08-08 00:05:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 216.948861ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XXX.X.X.........................................XX..X.X...] duration: 937.890649ms FAIL (2025-08-08 00:04:53 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 937.914916ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [.X............................................XXXXXXX...X...] duration: 1.770676593s FAIL (2025-08-08 00:04:51 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.770696592s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [X......................................XXX.X.XXX.X..XX.X.X..] duration: 279.909732ms FAIL (2025-08-08 00:04:50 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 279.936262ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..XXX.X......................................XXXXX.....X.X..] duration: 13.960904866s FAIL (2025-08-08 00:04:47 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 13.960934672s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 707.599042ms FAIL (2025-08-08 00:04:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 707.658093ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..........................................X...........X....] duration: 481.855912ms FAIL (2025-08-08 00:04:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 481.87638ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.........................................XXXXXXXXX...X......] duration: 176.57335ms FAIL (2025-08-08 00:04:24 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 176.604697ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.X......................................XXX.X.XXX.X..XX.X.X.] duration: 328.823089ms FAIL (2025-08-08 00:04:22 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 328.840352ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [..X......................................XXX.X.XXX.X..XX.X.X] duration: 30.000918003s FAIL (2025-08-08 00:04:21 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000936797s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X...........................................X.X..XXX.....X..] duration: 2.84425486s FAIL (2025-08-08 00:04:16 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.844283393s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [..X............................................XXXXXXX...X..] duration: 1.45709939s FAIL (2025-08-08 00:04:15 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.457117673s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X....................................................X.....] duration: 238.234428ms FAIL (2025-08-08 00:04:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 238.290621ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XXX.X.X.........................................XX..X.X..] duration: 1.414572095s FAIL (2025-08-08 00:04:06 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 1.414591582s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [....XXX.X.X.........................................XX..X.X.] duration: 41.693803924s FAIL (2025-08-08 00:04:04 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 41.693836968s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...XXX.X......................................XXXXX.....X.X.] duration: 356.858326ms FAIL (2025-08-08 00:04:02 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 356.8765ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XX.X.......X.] duration: 210.602082ms FAIL (2025-08-08 00:04:00 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 210.623512ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.39430717s FAIL (2025-08-08 00:03:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.394333761s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 5 out of 60 is failed (threshold=3) [X...............................................XX.X.......X] duration: 1m0.001333656s FAIL (2025-08-08 00:03:49 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.001360416s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..........................................XXXXXXXXX...X.....] duration: 166.395813ms FAIL (2025-08-08 00:03:46 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 166.418406ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 848.389628ms FAIL (2025-08-08 00:03:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 848.412522ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....XXX.X.X.........................................XX..X.X] duration: 1m0.000507248s FAIL (2025-08-08 00:03:15 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 1m0.000519199s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X..........................................X...........X...] duration: 385.682716ms FAIL (2025-08-08 00:03:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 385.701861ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X...........................................X.X..XXX.....X.] duration: 1.865165054s FAIL (2025-08-08 00:03:12 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.865191384s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [X..X......................................XXX.X.XXX.X..XX.X.] duration: 188.853981ms FAIL (2025-08-08 00:03:12 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 188.872566ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.X..X......................................XXX.X.XXX.X..XX.X] duration: 30.001174502s FAIL (2025-08-08 00:03:08 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00121678s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X....................................................X....] duration: 233.469685ms FAIL (2025-08-08 00:03:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 233.494911ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [X..X............................................XXXXXXX...X.] duration: 1.544932048s FAIL (2025-08-08 00:03:03 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.545252967s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [XX..X............................................XXXXXXX...X] duration: 1m0.000407812s FAIL (2025-08-08 00:03:01 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000418542s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X...........................................X.X..XXX.....X] duration: 30.001517906s FAIL (2025-08-08 00:02:58 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001537984s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.X..........................................X...........X..] duration: 651.943877ms FAIL (2025-08-08 00:02:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 651.961869ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...........................................XXXXXXXXX...X....] duration: 241.844949ms FAIL (2025-08-08 00:02:50 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 241.868032ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 177.817721ms FAIL (2025-08-08 00:02:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 177.971358ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X....XXX.X......................................XXXXX.....X.] duration: 503.327137ms FAIL (2025-08-08 00:02:43 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== . successful: 13 optimal: 13 errs: <nil> Aug 8 00:00:48.552480588 (7h34m2.13953683s) [upload=7750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:00:48.552487821 (7h34m2.139544062s) [upload=7750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:48.596293603 (7h34m2.183349845s) [upload=7750]: Done uploading segments... Aug 8 00:00:48.596306147 (7h34m2.183362389s) [upload=7750]: Upload finished. err: <nil> Aug 8 00:01:04.360824974 (7h34m17.947881195s) [upload=7753]: Starting upload Aug 8 00:01:04.360872293 (7h34m17.947928503s) [upload=7753]: Done starting upload Aug 8 00:01:04.360907538 (7h34m17.947963759s) [upload=7753]: Uploading segments... Aug 8 00:01:04.361092694 (7h34m17.948148915s) [upload=7753]: Got next segment. Inline: false Aug 8 00:01:04.361132418 (7h34m17.948188639s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:04.361138158 (7h34m17.94819437s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:01:04.384306121 (7h34m17.971362342s) [seg_pos={0 0}, upload=7753]: Begin upload segment... Aug 8 00:01:04.384337399 (7h34m17.97139362s) [seg_pos={0 0}, upload=7753]: Done begin upload segment. Aug 8 00:01:04.384352147 (7h34m17.971408358s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384363718 (7h34m17.97141994s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384394245 (7h34m17.971450456s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384420654 (7h34m17.971476876s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384672525 (7h34m17.971728766s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384680811 (7h34m17.971737021s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384893778 (7h34m17.971949989s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384905430 (7h34m17.97196164s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385111724 (7h34m17.972167945s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385321807 (7h34m17.972378028s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385547287 (7h34m17.972603499s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385780032 (7h34m17.972836253s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386010562 (7h34m17.973066783s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386253636 (7h34m17.973309857s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386512819 (7h34m17.97356903s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386763979 (7h34m17.9738202s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.481200832 (7h34m18.068257043s) [upload=7753]: Next returned nil segment Aug 8 00:01:04.481212825 (7h34m18.068269035s) [upload=7753]: Waiting for error group managing segments... Aug 8 00:01:04.493233591 (7h34m18.080289802s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494598709 (7h34m18.08165492s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494954183 (7h34m18.082010394s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.495060111 (7h34m18.082116332s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496576410 (7h34m18.083632621s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496907479 (7h34m18.083963689s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497177513 (7h34m18.084233724s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497538055 (7h34m18.084594266s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.499554820 (7h34m18.08661103s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501339521 (7h34m18.088395732s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501639971 (7h34m18.088696183s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.502941039 (7h34m18.089997261s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503363599 (7h34m18.0904198s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503371744 (7h34m18.090427954s) [seg_pos={0 0}, upload=7753]: Segment reached optimal threshold of 13 pieces. Aug 8 00:01:04.503387753 (7h34m18.090443955s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503507487 (7h34m18.090563698s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503586976 (7h34m18.090643187s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503596583 (7h34m18.090652794s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:01:04.503736805 (7h34m18.090793016s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:04.503742045 (7h34m18.090798256s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:04.558580310 (7h34m18.14563656s) [upload=7753]: Done uploading segments... Aug 8 00:01:04.558599866 (7h34m18.145656107s) [upload=7753]: Upload finished. err: <nil> Aug 8 00:02:02.017383460 (7h35m15.604439681s) [upload=7769]: Starting upload Aug 8 00:02:02.017467698 (7h35m15.604523899s) [upload=7769]: Done starting upload Aug 8 00:02:02.017580639 (7h35m15.604636861s) [upload=7769]: Uploading segments... Aug 8 00:02:02.017743643 (7h35m15.604799854s) [upload=7769]: Got next segment. Inline: false Aug 8 00:02:02.017795160 (7h35m15.60485137s) [upload=7769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:02.017800670 (7h35m15.604856881s) [upload=7769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:02:32.006066748 (7h35m45.593122959s) [upload=7769]: Next segment err: context deadline exceeded Aug 8 00:02:32.006146126 (7h35m45.593202338s) [upload=7769]: Done uploading segments... Aug 8 00:02:32.006171594 (7h35m45.593227815s) [upload=7769]: Upload finished. err: context deadline exceeded Duration: 503.355279ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X....XXX.X......................................XXXXX.....X] duration: 30.000979178s FAIL (2025-08-08 00:02:32 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== . successful: 13 optimal: 13 errs: <nil> Aug 8 00:00:48.552480588 (7h34m2.13953683s) [upload=7750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:00:48.552487821 (7h34m2.139544062s) [upload=7750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:48.596293603 (7h34m2.183349845s) [upload=7750]: Done uploading segments... Aug 8 00:00:48.596306147 (7h34m2.183362389s) [upload=7750]: Upload finished. err: <nil> Aug 8 00:01:04.360824974 (7h34m17.947881195s) [upload=7753]: Starting upload Aug 8 00:01:04.360872293 (7h34m17.947928503s) [upload=7753]: Done starting upload Aug 8 00:01:04.360907538 (7h34m17.947963759s) [upload=7753]: Uploading segments... Aug 8 00:01:04.361092694 (7h34m17.948148915s) [upload=7753]: Got next segment. Inline: false Aug 8 00:01:04.361132418 (7h34m17.948188639s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:04.361138158 (7h34m17.94819437s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:01:04.384306121 (7h34m17.971362342s) [seg_pos={0 0}, upload=7753]: Begin upload segment... Aug 8 00:01:04.384337399 (7h34m17.97139362s) [seg_pos={0 0}, upload=7753]: Done begin upload segment. Aug 8 00:01:04.384352147 (7h34m17.971408358s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384363718 (7h34m17.97141994s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384394245 (7h34m17.971450456s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384420654 (7h34m17.971476876s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384672525 (7h34m17.971728766s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384680811 (7h34m17.971737021s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384893778 (7h34m17.971949989s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384905430 (7h34m17.97196164s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385111724 (7h34m17.972167945s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385321807 (7h34m17.972378028s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385547287 (7h34m17.972603499s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385780032 (7h34m17.972836253s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386010562 (7h34m17.973066783s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386253636 (7h34m17.973309857s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386512819 (7h34m17.97356903s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386763979 (7h34m17.9738202s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.481200832 (7h34m18.068257043s) [upload=7753]: Next returned nil segment Aug 8 00:01:04.481212825 (7h34m18.068269035s) [upload=7753]: Waiting for error group managing segments... Aug 8 00:01:04.493233591 (7h34m18.080289802s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494598709 (7h34m18.08165492s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494954183 (7h34m18.082010394s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.495060111 (7h34m18.082116332s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496576410 (7h34m18.083632621s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496907479 (7h34m18.083963689s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497177513 (7h34m18.084233724s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497538055 (7h34m18.084594266s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.499554820 (7h34m18.08661103s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501339521 (7h34m18.088395732s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501639971 (7h34m18.088696183s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.502941039 (7h34m18.089997261s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503363599 (7h34m18.0904198s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503371744 (7h34m18.090427954s) [seg_pos={0 0}, upload=7753]: Segment reached optimal threshold of 13 pieces. Aug 8 00:01:04.503387753 (7h34m18.090443955s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503507487 (7h34m18.090563698s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503586976 (7h34m18.090643187s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503596583 (7h34m18.090652794s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:01:04.503736805 (7h34m18.090793016s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:04.503742045 (7h34m18.090798256s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:04.558580310 (7h34m18.14563656s) [upload=7753]: Done uploading segments... Aug 8 00:01:04.558599866 (7h34m18.145656107s) [upload=7753]: Upload finished. err: <nil> Aug 8 00:02:02.017383460 (7h35m15.604439681s) [upload=7769]: Starting upload Aug 8 00:02:02.017467698 (7h35m15.604523899s) [upload=7769]: Done starting upload Aug 8 00:02:02.017580639 (7h35m15.604636861s) [upload=7769]: Uploading segments... Aug 8 00:02:02.017743643 (7h35m15.604799854s) [upload=7769]: Got next segment. Inline: false Aug 8 00:02:02.017795160 (7h35m15.60485137s) [upload=7769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:02.017800670 (7h35m15.604856881s) [upload=7769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:02:32.006066748 (7h35m45.593122959s) [upload=7769]: Next segment err: context deadline exceeded Aug 8 00:02:32.006146126 (7h35m45.593202338s) [upload=7769]: Done uploading segments... Aug 8 00:02:32.006171594 (7h35m45.593227815s) [upload=7769]: Upload finished. err: context deadline exceeded Duration: 30.000997392s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 737.456795ms FAIL (2025-08-08 00:02:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 737.475931ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.X..........................................X...........X.] duration: 593.314537ms FAIL (2025-08-08 00:02:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 593.334164ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [......XXX.X.X.........................................XX..X.] duration: 28.294141937s FAIL (2025-08-08 00:02:15 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =7685]: Done starting upload Aug 7 23:56:26.112946671 (7h29m39.700002912s) [upload=7685]: Uploading segments... Aug 7 23:56:26.113307024 (7h29m39.700363265s) [upload=7685]: Got next segment. Inline: false Aug 7 23:56:26.113355876 (7h29m39.700412117s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:26.113362618 (7h29m39.700418859s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:41.416515391 (7h29m55.003571642s) [seg_pos={0 0}, upload=7685]: Begin upload segment... Aug 7 23:56:41.416597103 (7h29m55.003653344s) [seg_pos={0 0}, upload=7685]: Done begin upload segment. Aug 7 23:56:41.416637698 (7h29m55.003693939s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.416648018 (7h29m55.003704309s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417133103 (7h29m55.004189345s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417214976 (7h29m55.004271227s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417661369 (7h29m55.00471761s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417781764 (7h29m55.004838005s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418200426 (7h29m55.005256666s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418318265 (7h29m55.005374516s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418710699 (7h29m55.005766949s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418900132 (7h29m55.005956382s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419288596 (7h29m55.006344878s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419568088 (7h29m55.006624339s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419865433 (7h29m55.006921684s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420396514 (7h29m55.007452935s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420438683 (7h29m55.007494974s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.422079145 (7h29m55.009135386s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.423075114 (7h29m55.010131365s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.424123361 (7h29m55.011179632s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.543148900 (7h29m55.13020514s) [upload=7685]: Next returned nil segment Aug 7 23:56:41.543219551 (7h29m55.130275812s) [upload=7685]: Waiting for error group managing segments... Aug 7 23:56:41.555451953 (7h29m55.142508195s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.556831739 (7h29m55.14388798s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.558390238 (7h29m55.145446479s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.568256594 (7h29m55.155312844s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.579463192 (7h29m55.166519433s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.583078421 (7h29m55.170134671s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.584225170 (7h29m55.171281422s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.590665141 (7h29m55.177721381s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.611966581 (7h29m55.199022822s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.627727450 (7h29m55.214783701s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.645759177 (7h29m55.232815418s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.774571312 (7h29m55.361627554s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.876799657 (7h29m55.463855878s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000385207 (7h29m55.587441438s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000439059 (7h29m55.587495289s) [seg_pos={0 0}, upload=7685]: Segment reached optimal threshold of 14 pieces. Aug 7 23:56:42.000640734 (7h29m55.587696955s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000697911 (7h29m55.587754142s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dgidHuoyZFQZQyBAfQrBAn3FXjgePo8GjuzPyBijKp5xb4wk1): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000755489 (7h29m55.58781173s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AcLsSmVz9sRtuDLs47WoWYC66YSbr2aXBjzBdjayg22daQ2Hp): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000812144 (7h29m55.587868355s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fTtTieEUquaKNUNzqK98PkX7tVxjsiNeMLjz8kQJwdLne7d9R): protocol: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection Aug 7 23:56:42.000840427 (7h29m55.587896659s) [seg_pos={0 0}, upload=7685]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:56:42.000912191 (7h29m55.587968402s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:56:42.000916679 (7h29m55.5879729s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:47.368617454 (7h35m0.955673666s) [upload=7685]: Done uploading segments... Aug 8 00:01:47.368640257 (7h35m0.955696468s) [upload=7685]: Upload finished. err: metaclient: context deadline exceeded Duration: 28.294171742s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [..X..X......................................XXX.X.XXX.X..XX.] duration: 216.649791ms FAIL (2025-08-08 00:02:15 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 216.677382ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..X....................................................X...] duration: 222.974247ms FAIL (2025-08-08 00:02:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 223.000905ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.X...............................................XX.X.......] duration: 191.397549ms FAIL (2025-08-08 00:02:10 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Duration:* 191.423006ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X.X...........................................X.X..XXX.....] duration: 1.672497756s FAIL (2025-08-08 00:02:09 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.672521701s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [X..X..X......................................XXX.X.XXX.X..XX] duration: 30.000947251s FAIL (2025-08-08 00:02:02 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000968659s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [............................................XXXXXXXXX...X...] duration: 179.89504ms FAIL (2025-08-08 00:02:01 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 179.910209ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [XXX..X............................................XXXXXXX...] duration: 57.184568289s FAIL (2025-08-08 00:02:01 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 57.184586233s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.X..........................................X...........X] duration: 30.000505828s FAIL (2025-08-08 00:01:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000526206s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......XXX.X.X.........................................XX..X] duration: 5m21.270857701s FAIL (2025-08-08 00:01:47 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =7685]: Done starting upload Aug 7 23:56:26.112946671 (7h29m39.700002912s) [upload=7685]: Uploading segments... Aug 7 23:56:26.113307024 (7h29m39.700363265s) [upload=7685]: Got next segment. Inline: false Aug 7 23:56:26.113355876 (7h29m39.700412117s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:26.113362618 (7h29m39.700418859s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:41.416515391 (7h29m55.003571642s) [seg_pos={0 0}, upload=7685]: Begin upload segment... Aug 7 23:56:41.416597103 (7h29m55.003653344s) [seg_pos={0 0}, upload=7685]: Done begin upload segment. Aug 7 23:56:41.416637698 (7h29m55.003693939s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.416648018 (7h29m55.003704309s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417133103 (7h29m55.004189345s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417214976 (7h29m55.004271227s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417661369 (7h29m55.00471761s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417781764 (7h29m55.004838005s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418200426 (7h29m55.005256666s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418318265 (7h29m55.005374516s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418710699 (7h29m55.005766949s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418900132 (7h29m55.005956382s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419288596 (7h29m55.006344878s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419568088 (7h29m55.006624339s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419865433 (7h29m55.006921684s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420396514 (7h29m55.007452935s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420438683 (7h29m55.007494974s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.422079145 (7h29m55.009135386s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.423075114 (7h29m55.010131365s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.424123361 (7h29m55.011179632s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.543148900 (7h29m55.13020514s) [upload=7685]: Next returned nil segment Aug 7 23:56:41.543219551 (7h29m55.130275812s) [upload=7685]: Waiting for error group managing segments... Aug 7 23:56:41.555451953 (7h29m55.142508195s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.556831739 (7h29m55.14388798s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.558390238 (7h29m55.145446479s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.568256594 (7h29m55.155312844s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.579463192 (7h29m55.166519433s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.583078421 (7h29m55.170134671s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.584225170 (7h29m55.171281422s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.590665141 (7h29m55.177721381s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.611966581 (7h29m55.199022822s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.627727450 (7h29m55.214783701s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.645759177 (7h29m55.232815418s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.774571312 (7h29m55.361627554s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.876799657 (7h29m55.463855878s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000385207 (7h29m55.587441438s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000439059 (7h29m55.587495289s) [seg_pos={0 0}, upload=7685]: Segment reached optimal threshold of 14 pieces. Aug 7 23:56:42.000640734 (7h29m55.587696955s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000697911 (7h29m55.587754142s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dgidHuoyZFQZQyBAfQrBAn3FXjgePo8GjuzPyBijKp5xb4wk1): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000755489 (7h29m55.58781173s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AcLsSmVz9sRtuDLs47WoWYC66YSbr2aXBjzBdjayg22daQ2Hp): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000812144 (7h29m55.587868355s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fTtTieEUquaKNUNzqK98PkX7tVxjsiNeMLjz8kQJwdLne7d9R): protocol: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection Aug 7 23:56:42.000840427 (7h29m55.587896659s) [seg_pos={0 0}, upload=7685]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:56:42.000912191 (7h29m55.587968402s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:56:42.000916679 (7h29m55.5879729s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:47.368617454 (7h35m0.955673666s) [upload=7685]: Done uploading segments... Aug 8 00:01:47.368640257 (7h35m0.955696468s) [upload=7685]: Upload finished. err: metaclient: context deadline exceeded Duration: 5m21.270882235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 4 out of 60 is failed (threshold=3) [XX..X....................................................X..] duration: 241.809129ms FAIL (2025-08-08 00:01:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 241.832122ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 5 out of 60 is failed (threshold=3) [X.X...............................................XX.X......] duration: 222.7728ms FAIL (2025-08-08 00:01:34 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Duration:* 222.796614ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 163.392979ms FAIL (2025-08-08 00:01:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 163.418116ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)