Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [.......XXX.X.XXX.X..XX.X.X..................................] duration: 179.417972ms FAIL (2025-08-08 00:30:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 179.446686ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [............XX.X.......X....................................] duration: 157.853985ms FAIL (2025-08-08 00:30:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 157.878881ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................XX..X.X..................................] duration: 740.307109ms FAIL (2025-08-08 00:30:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 740.332717ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X.X..XXX.....X.....................................] duration: 2.123741647s FAIL (2025-08-08 00:30:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.123768266s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [............XXXXXXX...X.....................................] duration: 1.706493184s FAIL (2025-08-08 00:30:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.706511778s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X...........X....................................] duration: 570.78212ms FAIL (2025-08-08 00:30:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 570.807607ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 251.961615ms FAIL (2025-08-08 00:30:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 251.98595ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 201.736665ms FAIL (2025-08-08 00:30:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 201.897474ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.....XXXXXXXXX...X..........................................] duration: 270.996712ms FAIL (2025-08-08 00:30:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 271.02231ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [.............XXXXXXX...X....................................] duration: 1.637452739s FAIL (2025-08-08 00:29:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.637471955s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XX..X.X.................................] duration: 664.753453ms FAIL (2025-08-08 00:29:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 664.779832ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X...........X...................................] duration: 539.899759ms FAIL (2025-08-08 00:29:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 539.918094ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [............XXXXX.....X.X...................................] duration: 421.981478ms FAIL (2025-08-08 00:29:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 422.003799ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [........XXX.X.XXX.X..XX.X.X.................................] duration: 199.396125ms FAIL (2025-08-08 00:29:46 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 199.425199ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............XX.X.......X...................................] duration: 161.043651ms FAIL (2025-08-08 00:29:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 161.075089ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X.X..XXX.....X....................................] duration: 1.999105028s FAIL (2025-08-08 00:29:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.999135345s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [......XXXXXXXXX...X.........................................] duration: 268.307286ms FAIL (2025-08-08 00:29:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 268.335669ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 227.66672ms FAIL (2025-08-08 00:29:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 227.715229ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 204.92319ms FAIL (2025-08-08 00:29:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 204.981368ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............XX.X.......X..................................] duration: 356.648203ms FAIL (2025-08-08 00:29:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 356.681055ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............XXXXX.....X.X..................................] duration: 355.129172ms FAIL (2025-08-08 00:29:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 355.150472ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........X.X..XXX.....X...................................] duration: 1.897045138s FAIL (2025-08-08 00:29:06 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.897066848s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XX..X.X................................] duration: 728.781286ms FAIL (2025-08-08 00:28:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 728.802286ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [..............XXXXXXX...X...................................] duration: 1.538053897s FAIL (2025-08-08 00:28:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.538074506s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.......XXXXXXXXX...X........................................] duration: 226.656003ms FAIL (2025-08-08 00:28:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 226.672103ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 233.006817ms FAIL (2025-08-08 00:28:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 233.031172ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [.........XXX.X.XXX.X..XX.X.X................................] duration: 168.909612ms FAIL (2025-08-08 00:28:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 168.928698ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............XX.X.......X.................................] duration: 180.796299ms FAIL (2025-08-08 00:28:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 180.820405ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X...........X..................................] duration: 417.462883ms FAIL (2025-08-08 00:28:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 417.478714ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............X.X..XXX.....X..................................] duration: 1.754281277s FAIL (2025-08-08 00:28:35 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.754306543s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............XXXXX.....X.X.................................] duration: 388.866502ms FAIL (2025-08-08 00:28:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 388.888282ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 162.896338ms FAIL (2025-08-08 00:28:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 162.914924ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 207.280701ms FAIL (2025-08-08 00:28:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 207.303644ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [..........XXX.X.XXX.X..XX.X.X...............................] duration: 181.756521ms FAIL (2025-08-08 00:28:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 181.783792ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XX..X.X...............................] duration: 845.010564ms FAIL (2025-08-08 00:28:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 845.030331ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [........XXXXXXXXX...X.......................................] duration: 185.746444ms FAIL (2025-08-08 00:28:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 185.767654ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [................XX.X.......X................................] duration: 181.203787ms FAIL (2025-08-08 00:28:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 181.231179ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............XXXXX.....X.X................................] duration: 388.641512ms FAIL (2025-08-08 00:27:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 388.666059ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [...............XXXXXXX...X..................................] duration: 1.523196838s FAIL (2025-08-08 00:27:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.523223679s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 209.799704ms FAIL (2025-08-08 00:27:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 209.821454ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X...........X.................................] duration: 378.879651ms FAIL (2025-08-08 00:27:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 378.89999ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............X.X..XXX.....X.................................] duration: 1.833156512s FAIL (2025-08-08 00:27:40 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.833176409s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 180.678258ms FAIL (2025-08-08 00:27:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 180.754791ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X...........X................................] duration: 370.166809ms FAIL (2025-08-08 00:27:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 370.187819ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................XX.X.......X...............................] duration: 188.86661ms FAIL (2025-08-08 00:27:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 188.888089ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [................XXXXXXX...X.................................] duration: 1.602921717s FAIL (2025-08-08 00:27:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.602937665s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 486.457629ms FAIL (2025-08-08 00:27:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 486.475833ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...........XXX.X.XXX.X..XX.X.X..............................] duration: 198.774081ms FAIL (2025-08-08 00:27:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 198.786445ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XX..X.X..............................] duration: 555.333786ms FAIL (2025-08-08 00:27:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 555.361999ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 190.134458ms FAIL (2025-08-08 00:27:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 190.15744ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)