Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 25 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration: 30.000502812s FAIL (2025-08-07 21:31:48 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =5107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:56.715637130 (4h58m10.302693351s) [upload=5124]: Starting upload Aug 7 21:24:56.715691522 (4h58m10.302747743s) [upload=5124]: Done starting upload Aug 7 21:24:56.715738660 (4h58m10.302794881s) [upload=5124]: Uploading segments... Aug 7 21:24:56.715821674 (4h58m10.302877895s) [upload=5124]: Got next segment. Inline: true Aug 7 21:24:56.715825752 (4h58m10.302881973s) [upload=5124]: Waiting for error group managing segments... Aug 7 21:24:56.715858083 (4h58m10.302914294s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:56.715861810 (4h58m10.302918031s) [upload=5124]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:56.715864014 (4h58m10.302920235s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:26.698879559 (4h58m40.285935771s) [upload=5124]: Done uploading segments... Aug 7 21:25:26.698896120 (4h58m40.285952331s) [upload=5124]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:27.243732093 (4h58m40.830788304s) [upload=5132]: Starting upload Aug 7 21:25:27.243787377 (4h58m40.830843598s) [upload=5132]: Done starting upload Aug 7 21:25:27.243846396 (4h58m40.830902607s) [upload=5132]: Uploading segments... Aug 7 21:25:27.243883505 (4h58m40.830939727s) [upload=5132]: Got next segment. Inline: true Aug 7 21:25:27.243887303 (4h58m40.830943524s) [upload=5132]: Waiting for error group managing segments... Aug 7 21:25:27.243942507 (4h58m40.830998728s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:27.243946624 (4h58m40.831002845s) [upload=5132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:27.243948898 (4h58m40.831005119s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.223887428 (4h59m10.810943669s) [upload=5132]: Done uploading segments... Aug 7 21:25:57.223909498 (4h59m10.81096574s) [upload=5132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:29.142150756 (4h59m42.729206977s) [upload=5147]: Starting upload Aug 7 21:26:29.142194398 (4h59m42.729250619s) [upload=5147]: Done starting upload Aug 7 21:26:29.142233742 (4h59m42.729289963s) [upload=5147]: Uploading segments... Aug 7 21:26:29.142260401 (4h59m42.729316612s) [upload=5147]: Got next segment. Inline: true Aug 7 21:26:29.142263868 (4h59m42.729320089s) [upload=5147]: Waiting for error group managing segments... Aug 7 21:26:29.142293002 (4h59m42.729349213s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.142297080 (4h59m42.729353301s) [upload=5147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:29.142299354 (4h59m42.729355575s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:59.127042625 (5h0m12.714098866s) [upload=5147]: Done uploading segments... Aug 7 21:26:59.127117424 (5h0m12.714173665s) [upload=5147]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:10.814290180 (5h0m24.401346421s) [upload=5160]: Starting upload Aug 7 21:27:10.814371522 (5h0m24.401427764s) [upload=5160]: Done starting upload Aug 7 21:27:10.814438597 (5h0m24.401494839s) [upload=5160]: Uploading segments... Aug 7 21:27:10.814480476 (5h0m24.401536717s) [upload=5160]: Got next segment. Inline: true Aug 7 21:27:10.814485826 (5h0m24.401542067s) [upload=5160]: Waiting for error group managing segments... Aug 7 21:27:10.814529677 (5h0m24.401585918s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:10.814535198 (5h0m24.401591438s) [upload=5160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:10.814538994 (5h0m24.401595236s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:40.714926118 (5h0m54.301982339s) [upload=5160]: Done uploading segments... Aug 7 21:27:40.714942067 (5h0m54.301998278s) [upload=5160]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:40.796846333 (5h0m54.383902554s) [upload=5169]: Starting upload Aug 7 21:27:40.796900224 (5h0m54.383956435s) [upload=5169]: Done starting upload Aug 7 21:27:40.796945107 (5h0m54.384001318s) [upload=5169]: Uploading segments... Aug 7 21:27:40.796979782 (5h0m54.384036003s) [upload=5169]: Got next segment. Inline: true Aug 7 21:27:40.796983369 (5h0m54.38403959s) [upload=5169]: Waiting for error group managing segments... Aug 7 21:27:40.797013695 (5h0m54.384069907s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:40.797017642 (5h0m54.384073863s) [upload=5169]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:40.797020007 (5h0m54.384076218s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:10.771421400 (5h1m24.358477651s) [upload=5169]: Done uploading segments... Aug 7 21:28:10.771479779 (5h1m24.35853602s) [upload=5169]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:23.151092147 (5h1m36.738148388s) [upload=5182]: Starting upload Aug 7 21:28:23.151190951 (5h1m36.738247193s) [upload=5182]: Done starting upload Aug 7 21:28:23.151342555 (5h1m36.738398796s) [upload=5182]: Uploading segments... Aug 7 21:28:23.151386697 (5h1m36.738442937s) [upload=5182]: Got next segment. Inline: true Aug 7 21:28:23.151392588 (5h1m36.738448829s) [upload=5182]: Waiting for error group managing segments... Aug 7 21:28:23.151436370 (5h1m36.738492611s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:23.151441740 (5h1m36.738497981s) [upload=5182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:23.151445377 (5h1m36.738501618s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:53.132183214 (5h2m6.719239455s) [upload=5182]: Done uploading segments... Aug 7 21:28:53.132205376 (5h2m6.719261617s) [upload=5182]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:03.059565675 (5h2m16.646621886s) [upload=5190]: Starting upload Aug 7 21:29:03.059620448 (5h2m16.646676659s) [upload=5190]: Done starting upload Aug 7 21:29:03.059662505 (5h2m16.646718727s) [upload=5190]: Uploading segments... Aug 7 21:29:03.059702511 (5h2m16.646758732s) [upload=5190]: Got next segment. Inline: true Aug 7 21:29:03.059707350 (5h2m16.646763561s) [upload=5190]: Waiting for error group managing segments... Aug 7 21:29:03.059743357 (5h2m16.646799578s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:03.059748677 (5h2m16.646804898s) [upload=5190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:03.059752263 (5h2m16.646808474s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:33.039700593 (5h2m46.626756834s) [upload=5190]: Done uploading segments... Aug 7 21:29:33.039722394 (5h2m46.626778635s) [upload=5190]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:33.107560385 (5h2m46.694616626s) [upload=5202]: Starting upload Aug 7 21:29:33.107638791 (5h2m46.694695032s) [upload=5202]: Done starting upload Aug 7 21:29:33.107697620 (5h2m46.694753871s) [upload=5202]: Uploading segments... Aug 7 21:29:33.107747444 (5h2m46.694803705s) [upload=5202]: Got next segment. Inline: true Aug 7 21:29:33.107753225 (5h2m46.694809465s) [upload=5202]: Waiting for error group managing segments... Aug 7 21:29:33.107796575 (5h2m46.694852816s) [upload=5202]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:33.107802446 (5h2m46.694858697s) [upload=5202]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:33.107805812 (5h2m46.694862053s) [upload=5202]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:03.094304693 (5h3m16.681360904s) [upload=5202]: Done uploading segments... Aug 7 21:30:03.094320953 (5h3m16.681377174s) [upload=5202]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:30:47.146001890 (5h4m0.733058111s) [upload=5219]: Starting upload Aug 7 21:30:47.146103910 (5h4m0.733160131s) [upload=5219]: Done starting upload Aug 7 21:30:47.146153031 (5h4m0.733209242s) [upload=5219]: Uploading segments... Aug 7 21:30:47.146191593 (5h4m0.733247814s) [upload=5219]: Got next segment. Inline: true Aug 7 21:30:47.146196132 (5h4m0.733252343s) [upload=5219]: Waiting for error group managing segments... Aug 7 21:30:47.146229184 (5h4m0.733285395s) [upload=5219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:47.146233180 (5h4m0.733289392s) [upload=5219]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:30:47.146235815 (5h4m0.733292036s) [upload=5219]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:31:17.134124198 (5h4m30.721180449s) [upload=5219]: Done uploading segments... Aug 7 21:31:17.134145378 (5h4m30.721201618s) [upload=5219]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:31:18.629774505 (5h4m32.216830716s) [upload=5229]: Starting upload Aug 7 21:31:18.629831501 (5h4m32.216887712s) [upload=5229]: Done starting upload Aug 7 21:31:18.629882436 (5h4m32.216938658s) [upload=5229]: Uploading segments... Aug 7 21:31:18.629913235 (5h4m32.216969445s) [upload=5229]: Got next segment. Inline: true Aug 7 21:31:18.629916951 (5h4m32.216973162s) [upload=5229]: Waiting for error group managing segments... Aug 7 21:31:18.629950052 (5h4m32.217006263s) [upload=5229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:31:18.629953899 (5h4m32.21701012s) [upload=5229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:31:18.629956264 (5h4m32.217012475s) [upload=5229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:31:48.616363445 (5h5m2.203419657s) [upload=5229]: Done uploading segments... Aug 7 21:31:48.616374867 (5h5m2.203431078s) [upload=5229]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000515154s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 23 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration: 30.000627836s FAIL (2025-08-07 21:31:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000638647s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 30.001158631s FAIL (2025-08-07 21:31:41 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 53985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Aug 7 21:23:50.132503139 (4h57m3.719559361s) [upload=5111]: Starting upload Aug 7 21:23:50.132542101 (4h57m3.719598322s) [upload=5111]: Done starting upload Aug 7 21:23:50.132574512 (4h57m3.719630733s) [upload=5111]: Uploading segments... Aug 7 21:23:50.132772312 (4h57m3.719828533s) [upload=5111]: Got next segment. Inline: false Aug 7 21:23:50.132810694 (4h57m3.719866904s) [upload=5111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:50.132814390 (4h57m3.719870611s) [upload=5111]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:20.119434924 (4h57m33.706491135s) [upload=5111]: Next segment err: context deadline exceeded Aug 7 21:24:20.119485478 (4h57m33.706541699s) [upload=5111]: Done uploading segments... Aug 7 21:24:20.119497891 (4h57m33.706554102s) [upload=5111]: Upload finished. err: context deadline exceeded Aug 7 21:24:45.494243280 (4h57m59.081299501s) [upload=5120]: Starting upload Aug 7 21:24:45.494313381 (4h57m59.081369612s) [upload=5120]: Done starting upload Aug 7 21:24:45.494474943 (4h57m59.081531184s) [upload=5120]: Uploading segments... Aug 7 21:24:45.494627989 (4h57m59.08168422s) [upload=5120]: Got next segment. Inline: false Aug 7 21:24:45.494663625 (4h57m59.081719856s) [upload=5120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:45.494668073 (4h57m59.081724294s) [upload=5120]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:15.480538548 (4h58m29.067594769s) [upload=5120]: Next segment err: context deadline exceeded Aug 7 21:25:15.480665896 (4h58m29.067722117s) [upload=5120]: Done uploading segments... Aug 7 21:25:15.480717081 (4h58m29.067773302s) [upload=5120]: Upload finished. err: context deadline exceeded Aug 7 21:25:15.571225478 (4h58m29.158281699s) [upload=5129]: Starting upload Aug 7 21:25:15.571281894 (4h58m29.158338105s) [upload=5129]: Done starting upload Aug 7 21:25:15.571359468 (4h58m29.158415689s) [upload=5129]: Uploading segments... Aug 7 21:25:15.571564180 (4h58m29.158620401s) [upload=5129]: Got next segment. Inline: false Aug 7 21:25:15.571622810 (4h58m29.158679021s) [upload=5129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.571628479 (4h58m29.1586847s) [upload=5129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:45.547010083 (4h58m59.134066294s) [upload=5129]: Next segment err: context deadline exceeded Aug 7 21:25:45.547084462 (4h58m59.134140683s) [upload=5129]: Done uploading segments... Aug 7 21:25:45.547107044 (4h58m59.134163266s) [upload=5129]: Upload finished. err: context deadline exceeded Aug 7 21:25:45.641521062 (4h58m59.228577283s) [upload=5137]: Starting upload Aug 7 21:25:45.641578831 (4h58m59.228635041s) [upload=5137]: Done starting upload Aug 7 21:25:45.641631238 (4h58m59.228687459s) [upload=5137]: Uploading segments... Aug 7 21:25:45.641794333 (4h58m59.228850554s) [upload=5137]: Got next segment. Inline: false Aug 7 21:25:45.641818608 (4h58m59.228874829s) [upload=5137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.641822736 (4h58m59.228878947s) [upload=5137]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:15.630688562 (4h59m29.217744793s) [upload=5137]: Next segment err: context deadline exceeded Aug 7 21:26:15.630873647 (4h59m29.217929858s) [upload=5137]: Done uploading segments... Aug 7 21:26:15.630892703 (4h59m29.217948914s) [upload=5137]: Upload finished. err: context deadline exceeded Aug 7 21:26:29.805240268 (4h59m43.392296479s) [upload=5148]: Starting upload Aug 7 21:26:29.805289190 (4h59m43.392345411s) [upload=5148]: Done starting upload Aug 7 21:26:29.805393004 (4h59m43.392449225s) [upload=5148]: Uploading segments... Aug 7 21:26:29.805516605 (4h59m43.392572816s) [upload=5148]: Got next segment. Inline: false Aug 7 21:26:29.805549756 (4h59m43.392605967s) [upload=5148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.805553633 (4h59m43.392609854s) [upload=5148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:59.790298417 (5h0m13.377354628s) [upload=5148]: Next segment err: context deadline exceeded Aug 7 21:26:59.790450801 (5h0m13.377507012s) [upload=5148]: Done uploading segments... Aug 7 21:26:59.790479374 (5h0m13.377535595s) [upload=5148]: Upload finished. err: context deadline exceeded Aug 7 21:26:59.854535904 (5h0m13.441592145s) [upload=5156]: Starting upload Aug 7 21:26:59.854627225 (5h0m13.441683465s) [upload=5156]: Done starting upload Aug 7 21:26:59.854723153 (5h0m13.441779394s) [upload=5156]: Uploading segments... Aug 7 21:26:59.855007455 (5h0m13.442063705s) [upload=5156]: Got next segment. Inline: false Aug 7 21:26:59.855059541 (5h0m13.442115782s) [upload=5156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:59.855096871 (5h0m13.442153123s) [upload=5156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:27:29.841757946 (5h0m43.428814187s) [upload=5156]: Next segment err: context deadline exceeded Aug 7 21:27:29.841960274 (5h0m43.429016514s) [upload=5156]: Done uploading segments... Aug 7 21:27:29.842007351 (5h0m43.429063593s) [upload=5156]: Upload finished. err: context deadline exceeded Aug 7 21:28:06.370955849 (5h1m19.95801209s) [upload=5175]: Starting upload Aug 7 21:28:06.371030067 (5h1m19.958086308s) [upload=5175]: Done starting upload Aug 7 21:28:06.371151734 (5h1m19.958207975s) [upload=5175]: Uploading segments... Aug 7 21:28:06.371451254 (5h1m19.958507495s) [upload=5175]: Got next segment. Inline: false Aug 7 21:28:06.371506767 (5h1m19.958563008s) [upload=5175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:06.371512959 (5h1m19.958569209s) [upload=5175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:28:36.358597390 (5h1m49.945653632s) [upload=5175]: Next segment err: context deadline exceeded Aug 7 21:28:36.358673913 (5h1m49.945730154s) [upload=5175]: Done uploading segments... Aug 7 21:28:36.358704921 (5h1m49.945761162s) [upload=5175]: Upload finished. err: context deadline exceeded Aug 7 21:29:01.102431480 (5h2m14.689487702s) [upload=5189]: Starting upload Aug 7 21:29:01.102546615 (5h2m14.689602836s) [upload=5189]: Done starting upload Aug 7 21:29:01.102685454 (5h2m14.689741675s) [upload=5189]: Uploading segments... Aug 7 21:29:01.102866763 (5h2m14.689922984s) [upload=5189]: Got next segment. Inline: false Aug 7 21:29:01.102903291 (5h2m14.689959502s) [upload=5189]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:01.102907839 (5h2m14.68996406s) [upload=5189]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:29:31.087349640 (5h2m44.67440585s) [upload=5189]: Next segment err: context deadline exceeded Aug 7 21:29:31.087429919 (5h2m44.67448613s) [upload=5189]: Done uploading segments... Aug 7 21:29:31.087444957 (5h2m44.674501178s) [upload=5189]: Upload finished. err: context deadline exceeded Aug 7 21:30:07.260405000 (5h3m20.847461241s) [upload=5206]: Starting upload Aug 7 21:30:07.260495098 (5h3m20.84755135s) [upload=5206]: Done starting upload Aug 7 21:30:07.260642223 (5h3m20.847698474s) [upload=5206]: Uploading segments... Aug 7 21:30:07.260927024 (5h3m20.847983276s) [upload=5206]: Got next segment. Inline: false Aug 7 21:30:07.260984843 (5h3m20.848041084s) [upload=5206]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:07.260994751 (5h3m20.848050993s) [upload=5206]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:30:37.232137150 (5h3m50.81919336s) [upload=5206]: Next segment err: context deadline exceeded Aug 7 21:30:37.232307578 (5h3m50.819363789s) [upload=5206]: Done uploading segments... Aug 7 21:30:37.232355838 (5h3m50.819412059s) [upload=5206]: Upload finished. err: context deadline exceeded Aug 7 21:30:41.838696318 (5h3m55.425752559s) [upload=5217]: Starting upload Aug 7 21:30:41.838781527 (5h3m55.425837767s) [upload=5217]: Done starting upload Aug 7 21:30:41.838903785 (5h3m55.425960006s) [upload=5217]: Uploading segments... Aug 7 21:30:41.839179650 (5h3m55.426235901s) [upload=5217]: Got next segment. Inline: false Aug 7 21:30:41.839223772 (5h3m55.426280023s) [upload=5217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:41.839229273 (5h3m55.426285513s) [upload=5217]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:31:11.826385247 (5h4m25.413441468s) [upload=5217]: Next segment err: context deadline exceeded Aug 7 21:31:11.826544515 (5h4m25.413600736s) [upload=5217]: Done uploading segments... Aug 7 21:31:11.826568178 (5h4m25.413624399s) [upload=5217]: Upload finished. err: context deadline exceeded Aug 7 21:31:11.924007243 (5h4m25.511063484s) [upload=5227]: Starting upload Aug 7 21:31:11.924116868 (5h4m25.511173108s) [upload=5227]: Done starting upload Aug 7 21:31:11.924293237 (5h4m25.511349458s) [upload=5227]: Uploading segments... Aug 7 21:31:11.924515282 (5h4m25.511571553s) [upload=5227]: Got next segment. Inline: false Aug 7 21:31:11.924580593 (5h4m25.511636844s) [upload=5227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:31:11.924591073 (5h4m25.511647324s) [upload=5227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:31:41.901407105 (5h4m55.488463316s) [upload=5227]: Next segment err: context deadline exceeded Aug 7 21:31:41.901545804 (5h4m55.488602015s) [upload=5227]: Done uploading segments... Aug 7 21:31:41.901568547 (5h4m55.488624747s) [upload=5227]: Upload finished. err: context deadline exceeded Duration: 30.001171786s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 18 out of 60 is failed (threshold=3) [.........................................XX.XXXXXXXXXXXXXXXX] duration: 1m0.001053904s FAIL (2025-08-07 21:31:32 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001069402s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 30.000574193s FAIL (2025-08-07 21:31:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00061458s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration: 30.001563678s FAIL (2025-08-07 21:31:17 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== load finished. err: metaclient: context deadline exceeded Aug 7 21:23:44.852200021 (4h56m58.439256242s) [upload=5107]: Starting upload Aug 7 21:23:44.852251448 (4h56m58.439307669s) [upload=5107]: Done starting upload Aug 7 21:23:44.852291703 (4h56m58.439347924s) [upload=5107]: Uploading segments... Aug 7 21:23:44.852320757 (4h56m58.439376978s) [upload=5107]: Got next segment. Inline: true Aug 7 21:23:44.852324224 (4h56m58.439380445s) [upload=5107]: Waiting for error group managing segments... Aug 7 21:23:44.852355542 (4h56m58.439411753s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:44.852359279 (4h56m58.4394155s) [upload=5107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:44.852361614 (4h56m58.439417824s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.835050939 (4h57m28.42210715s) [upload=5107]: Done uploading segments... Aug 7 21:24:14.835115480 (4h57m28.422171701s) [upload=5107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:56.715637130 (4h58m10.302693351s) [upload=5124]: Starting upload Aug 7 21:24:56.715691522 (4h58m10.302747743s) [upload=5124]: Done starting upload Aug 7 21:24:56.715738660 (4h58m10.302794881s) [upload=5124]: Uploading segments... Aug 7 21:24:56.715821674 (4h58m10.302877895s) [upload=5124]: Got next segment. Inline: true Aug 7 21:24:56.715825752 (4h58m10.302881973s) [upload=5124]: Waiting for error group managing segments... Aug 7 21:24:56.715858083 (4h58m10.302914294s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:56.715861810 (4h58m10.302918031s) [upload=5124]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:56.715864014 (4h58m10.302920235s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:26.698879559 (4h58m40.285935771s) [upload=5124]: Done uploading segments... Aug 7 21:25:26.698896120 (4h58m40.285952331s) [upload=5124]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:27.243732093 (4h58m40.830788304s) [upload=5132]: Starting upload Aug 7 21:25:27.243787377 (4h58m40.830843598s) [upload=5132]: Done starting upload Aug 7 21:25:27.243846396 (4h58m40.830902607s) [upload=5132]: Uploading segments... Aug 7 21:25:27.243883505 (4h58m40.830939727s) [upload=5132]: Got next segment. Inline: true Aug 7 21:25:27.243887303 (4h58m40.830943524s) [upload=5132]: Waiting for error group managing segments... Aug 7 21:25:27.243942507 (4h58m40.830998728s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:27.243946624 (4h58m40.831002845s) [upload=5132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:27.243948898 (4h58m40.831005119s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.223887428 (4h59m10.810943669s) [upload=5132]: Done uploading segments... Aug 7 21:25:57.223909498 (4h59m10.81096574s) [upload=5132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:29.142150756 (4h59m42.729206977s) [upload=5147]: Starting upload Aug 7 21:26:29.142194398 (4h59m42.729250619s) [upload=5147]: Done starting upload Aug 7 21:26:29.142233742 (4h59m42.729289963s) [upload=5147]: Uploading segments... Aug 7 21:26:29.142260401 (4h59m42.729316612s) [upload=5147]: Got next segment. Inline: true Aug 7 21:26:29.142263868 (4h59m42.729320089s) [upload=5147]: Waiting for error group managing segments... Aug 7 21:26:29.142293002 (4h59m42.729349213s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.142297080 (4h59m42.729353301s) [upload=5147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:29.142299354 (4h59m42.729355575s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:59.127042625 (5h0m12.714098866s) [upload=5147]: Done uploading segments... Aug 7 21:26:59.127117424 (5h0m12.714173665s) [upload=5147]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:10.814290180 (5h0m24.401346421s) [upload=5160]: Starting upload Aug 7 21:27:10.814371522 (5h0m24.401427764s) [upload=5160]: Done starting upload Aug 7 21:27:10.814438597 (5h0m24.401494839s) [upload=5160]: Uploading segments... Aug 7 21:27:10.814480476 (5h0m24.401536717s) [upload=5160]: Got next segment. Inline: true Aug 7 21:27:10.814485826 (5h0m24.401542067s) [upload=5160]: Waiting for error group managing segments... Aug 7 21:27:10.814529677 (5h0m24.401585918s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:10.814535198 (5h0m24.401591438s) [upload=5160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:10.814538994 (5h0m24.401595236s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:40.714926118 (5h0m54.301982339s) [upload=5160]: Done uploading segments... Aug 7 21:27:40.714942067 (5h0m54.301998278s) [upload=5160]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:40.796846333 (5h0m54.383902554s) [upload=5169]: Starting upload Aug 7 21:27:40.796900224 (5h0m54.383956435s) [upload=5169]: Done starting upload Aug 7 21:27:40.796945107 (5h0m54.384001318s) [upload=5169]: Uploading segments... Aug 7 21:27:40.796979782 (5h0m54.384036003s) [upload=5169]: Got next segment. Inline: true Aug 7 21:27:40.796983369 (5h0m54.38403959s) [upload=5169]: Waiting for error group managing segments... Aug 7 21:27:40.797013695 (5h0m54.384069907s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:40.797017642 (5h0m54.384073863s) [upload=5169]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:40.797020007 (5h0m54.384076218s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:10.771421400 (5h1m24.358477651s) [upload=5169]: Done uploading segments... Aug 7 21:28:10.771479779 (5h1m24.35853602s) [upload=5169]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:23.151092147 (5h1m36.738148388s) [upload=5182]: Starting upload Aug 7 21:28:23.151190951 (5h1m36.738247193s) [upload=5182]: Done starting upload Aug 7 21:28:23.151342555 (5h1m36.738398796s) [upload=5182]: Uploading segments... Aug 7 21:28:23.151386697 (5h1m36.738442937s) [upload=5182]: Got next segment. Inline: true Aug 7 21:28:23.151392588 (5h1m36.738448829s) [upload=5182]: Waiting for error group managing segments... Aug 7 21:28:23.151436370 (5h1m36.738492611s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:23.151441740 (5h1m36.738497981s) [upload=5182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:23.151445377 (5h1m36.738501618s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:53.132183214 (5h2m6.719239455s) [upload=5182]: Done uploading segments... Aug 7 21:28:53.132205376 (5h2m6.719261617s) [upload=5182]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:03.059565675 (5h2m16.646621886s) [upload=5190]: Starting upload Aug 7 21:29:03.059620448 (5h2m16.646676659s) [upload=5190]: Done starting upload Aug 7 21:29:03.059662505 (5h2m16.646718727s) [upload=5190]: Uploading segments... Aug 7 21:29:03.059702511 (5h2m16.646758732s) [upload=5190]: Got next segment. Inline: true Aug 7 21:29:03.059707350 (5h2m16.646763561s) [upload=5190]: Waiting for error group managing segments... Aug 7 21:29:03.059743357 (5h2m16.646799578s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:03.059748677 (5h2m16.646804898s) [upload=5190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:03.059752263 (5h2m16.646808474s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:33.039700593 (5h2m46.626756834s) [upload=5190]: Done uploading segments... Aug 7 21:29:33.039722394 (5h2m46.626778635s) [upload=5190]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:33.107560385 (5h2m46.694616626s) [upload=5202]: Starting upload Aug 7 21:29:33.107638791 (5h2m46.694695032s) [upload=5202]: Done starting upload Aug 7 21:29:33.107697620 (5h2m46.694753871s) [upload=5202]: Uploading segments... Aug 7 21:29:33.107747444 (5h2m46.694803705s) [upload=5202]: Got next segment. Inline: true Aug 7 21:29:33.107753225 (5h2m46.694809465s) [upload=5202]: Waiting for error group managing segments... Aug 7 21:29:33.107796575 (5h2m46.694852816s) [upload=5202]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:33.107802446 (5h2m46.694858697s) [upload=5202]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:33.107805812 (5h2m46.694862053s) [upload=5202]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:03.094304693 (5h3m16.681360904s) [upload=5202]: Done uploading segments... Aug 7 21:30:03.094320953 (5h3m16.681377174s) [upload=5202]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:30:47.146001890 (5h4m0.733058111s) [upload=5219]: Starting upload Aug 7 21:30:47.146103910 (5h4m0.733160131s) [upload=5219]: Done starting upload Aug 7 21:30:47.146153031 (5h4m0.733209242s) [upload=5219]: Uploading segments... Aug 7 21:30:47.146191593 (5h4m0.733247814s) [upload=5219]: Got next segment. Inline: true Aug 7 21:30:47.146196132 (5h4m0.733252343s) [upload=5219]: Waiting for error group managing segments... Aug 7 21:30:47.146229184 (5h4m0.733285395s) [upload=5219]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:47.146233180 (5h4m0.733289392s) [upload=5219]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:30:47.146235815 (5h4m0.733292036s) [upload=5219]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:31:17.134124198 (5h4m30.721180449s) [upload=5219]: Done uploading segments... Aug 7 21:31:17.134145378 (5h4m30.721201618s) [upload=5219]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001583095s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 609.172989ms FAIL (2025-08-07 21:31:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 609.193508ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 30.001298967s FAIL (2025-08-07 21:31:11 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Aug 7 21:23:20.066472018 (4h56m33.653528259s) [upload=5100]: Starting upload Aug 7 21:23:20.066538723 (4h56m33.653594965s) [upload=5100]: Done starting upload Aug 7 21:23:20.066650672 (4h56m33.653706913s) [upload=5100]: Uploading segments... Aug 7 21:23:20.066882696 (4h56m33.653938936s) [upload=5100]: Got next segment. Inline: false Aug 7 21:23:20.066929522 (4h56m33.653985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Aug 7 21:23:50.132503139 (4h57m3.719559361s) [upload=5111]: Starting upload Aug 7 21:23:50.132542101 (4h57m3.719598322s) [upload=5111]: Done starting upload Aug 7 21:23:50.132574512 (4h57m3.719630733s) [upload=5111]: Uploading segments... Aug 7 21:23:50.132772312 (4h57m3.719828533s) [upload=5111]: Got next segment. Inline: false Aug 7 21:23:50.132810694 (4h57m3.719866904s) [upload=5111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:50.132814390 (4h57m3.719870611s) [upload=5111]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:20.119434924 (4h57m33.706491135s) [upload=5111]: Next segment err: context deadline exceeded Aug 7 21:24:20.119485478 (4h57m33.706541699s) [upload=5111]: Done uploading segments... Aug 7 21:24:20.119497891 (4h57m33.706554102s) [upload=5111]: Upload finished. err: context deadline exceeded Aug 7 21:24:45.494243280 (4h57m59.081299501s) [upload=5120]: Starting upload Aug 7 21:24:45.494313381 (4h57m59.081369612s) [upload=5120]: Done starting upload Aug 7 21:24:45.494474943 (4h57m59.081531184s) [upload=5120]: Uploading segments... Aug 7 21:24:45.494627989 (4h57m59.08168422s) [upload=5120]: Got next segment. Inline: false Aug 7 21:24:45.494663625 (4h57m59.081719856s) [upload=5120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:45.494668073 (4h57m59.081724294s) [upload=5120]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:15.480538548 (4h58m29.067594769s) [upload=5120]: Next segment err: context deadline exceeded Aug 7 21:25:15.480665896 (4h58m29.067722117s) [upload=5120]: Done uploading segments... Aug 7 21:25:15.480717081 (4h58m29.067773302s) [upload=5120]: Upload finished. err: context deadline exceeded Aug 7 21:25:15.571225478 (4h58m29.158281699s) [upload=5129]: Starting upload Aug 7 21:25:15.571281894 (4h58m29.158338105s) [upload=5129]: Done starting upload Aug 7 21:25:15.571359468 (4h58m29.158415689s) [upload=5129]: Uploading segments... Aug 7 21:25:15.571564180 (4h58m29.158620401s) [upload=5129]: Got next segment. Inline: false Aug 7 21:25:15.571622810 (4h58m29.158679021s) [upload=5129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.571628479 (4h58m29.1586847s) [upload=5129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:45.547010083 (4h58m59.134066294s) [upload=5129]: Next segment err: context deadline exceeded Aug 7 21:25:45.547084462 (4h58m59.134140683s) [upload=5129]: Done uploading segments... Aug 7 21:25:45.547107044 (4h58m59.134163266s) [upload=5129]: Upload finished. err: context deadline exceeded Aug 7 21:25:45.641521062 (4h58m59.228577283s) [upload=5137]: Starting upload Aug 7 21:25:45.641578831 (4h58m59.228635041s) [upload=5137]: Done starting upload Aug 7 21:25:45.641631238 (4h58m59.228687459s) [upload=5137]: Uploading segments... Aug 7 21:25:45.641794333 (4h58m59.228850554s) [upload=5137]: Got next segment. Inline: false Aug 7 21:25:45.641818608 (4h58m59.228874829s) [upload=5137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.641822736 (4h58m59.228878947s) [upload=5137]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:15.630688562 (4h59m29.217744793s) [upload=5137]: Next segment err: context deadline exceeded Aug 7 21:26:15.630873647 (4h59m29.217929858s) [upload=5137]: Done uploading segments... Aug 7 21:26:15.630892703 (4h59m29.217948914s) [upload=5137]: Upload finished. err: context deadline exceeded Aug 7 21:26:29.805240268 (4h59m43.392296479s) [upload=5148]: Starting upload Aug 7 21:26:29.805289190 (4h59m43.392345411s) [upload=5148]: Done starting upload Aug 7 21:26:29.805393004 (4h59m43.392449225s) [upload=5148]: Uploading segments... Aug 7 21:26:29.805516605 (4h59m43.392572816s) [upload=5148]: Got next segment. Inline: false Aug 7 21:26:29.805549756 (4h59m43.392605967s) [upload=5148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.805553633 (4h59m43.392609854s) [upload=5148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:59.790298417 (5h0m13.377354628s) [upload=5148]: Next segment err: context deadline exceeded Aug 7 21:26:59.790450801 (5h0m13.377507012s) [upload=5148]: Done uploading segments... Aug 7 21:26:59.790479374 (5h0m13.377535595s) [upload=5148]: Upload finished. err: context deadline exceeded Aug 7 21:26:59.854535904 (5h0m13.441592145s) [upload=5156]: Starting upload Aug 7 21:26:59.854627225 (5h0m13.441683465s) [upload=5156]: Done starting upload Aug 7 21:26:59.854723153 (5h0m13.441779394s) [upload=5156]: Uploading segments... Aug 7 21:26:59.855007455 (5h0m13.442063705s) [upload=5156]: Got next segment. Inline: false Aug 7 21:26:59.855059541 (5h0m13.442115782s) [upload=5156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:59.855096871 (5h0m13.442153123s) [upload=5156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:27:29.841757946 (5h0m43.428814187s) [upload=5156]: Next segment err: context deadline exceeded Aug 7 21:27:29.841960274 (5h0m43.429016514s) [upload=5156]: Done uploading segments... Aug 7 21:27:29.842007351 (5h0m43.429063593s) [upload=5156]: Upload finished. err: context deadline exceeded Aug 7 21:28:06.370955849 (5h1m19.95801209s) [upload=5175]: Starting upload Aug 7 21:28:06.371030067 (5h1m19.958086308s) [upload=5175]: Done starting upload Aug 7 21:28:06.371151734 (5h1m19.958207975s) [upload=5175]: Uploading segments... Aug 7 21:28:06.371451254 (5h1m19.958507495s) [upload=5175]: Got next segment. Inline: false Aug 7 21:28:06.371506767 (5h1m19.958563008s) [upload=5175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:06.371512959 (5h1m19.958569209s) [upload=5175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:28:36.358597390 (5h1m49.945653632s) [upload=5175]: Next segment err: context deadline exceeded Aug 7 21:28:36.358673913 (5h1m49.945730154s) [upload=5175]: Done uploading segments... Aug 7 21:28:36.358704921 (5h1m49.945761162s) [upload=5175]: Upload finished. err: context deadline exceeded Aug 7 21:29:01.102431480 (5h2m14.689487702s) [upload=5189]: Starting upload Aug 7 21:29:01.102546615 (5h2m14.689602836s) [upload=5189]: Done starting upload Aug 7 21:29:01.102685454 (5h2m14.689741675s) [upload=5189]: Uploading segments... Aug 7 21:29:01.102866763 (5h2m14.689922984s) [upload=5189]: Got next segment. Inline: false Aug 7 21:29:01.102903291 (5h2m14.689959502s) [upload=5189]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:01.102907839 (5h2m14.68996406s) [upload=5189]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:29:31.087349640 (5h2m44.67440585s) [upload=5189]: Next segment err: context deadline exceeded Aug 7 21:29:31.087429919 (5h2m44.67448613s) [upload=5189]: Done uploading segments... Aug 7 21:29:31.087444957 (5h2m44.674501178s) [upload=5189]: Upload finished. err: context deadline exceeded Aug 7 21:30:07.260405000 (5h3m20.847461241s) [upload=5206]: Starting upload Aug 7 21:30:07.260495098 (5h3m20.84755135s) [upload=5206]: Done starting upload Aug 7 21:30:07.260642223 (5h3m20.847698474s) [upload=5206]: Uploading segments... Aug 7 21:30:07.260927024 (5h3m20.847983276s) [upload=5206]: Got next segment. Inline: false Aug 7 21:30:07.260984843 (5h3m20.848041084s) [upload=5206]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:07.260994751 (5h3m20.848050993s) [upload=5206]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:30:37.232137150 (5h3m50.81919336s) [upload=5206]: Next segment err: context deadline exceeded Aug 7 21:30:37.232307578 (5h3m50.819363789s) [upload=5206]: Done uploading segments... Aug 7 21:30:37.232355838 (5h3m50.819412059s) [upload=5206]: Upload finished. err: context deadline exceeded Aug 7 21:30:41.838696318 (5h3m55.425752559s) [upload=5217]: Starting upload Aug 7 21:30:41.838781527 (5h3m55.425837767s) [upload=5217]: Done starting upload Aug 7 21:30:41.838903785 (5h3m55.425960006s) [upload=5217]: Uploading segments... Aug 7 21:30:41.839179650 (5h3m55.426235901s) [upload=5217]: Got next segment. Inline: false Aug 7 21:30:41.839223772 (5h3m55.426280023s) [upload=5217]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:41.839229273 (5h3m55.426285513s) [upload=5217]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:31:11.826385247 (5h4m25.413441468s) [upload=5217]: Next segment err: context deadline exceeded Aug 7 21:31:11.826544515 (5h4m25.413600736s) [upload=5217]: Done uploading segments... Aug 7 21:31:11.826568178 (5h4m25.413624399s) [upload=5217]: Upload finished. err: context deadline exceeded Duration: 30.001324483s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration: 9m24.674645578s FAIL (2025-08-07 21:31:08 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 21:22:31.352295676 (4h55m44.939351886s) [seg_pos={0 0}, upload=5078]: Begin upload segment... Aug 7 21:22:31.352327955 (4h55m44.939384166s) [seg_pos={0 0}, upload=5078]: Done begin upload segment. Aug 7 21:22:31.352351249 (4h55m44.93940747s) [noise=true , address=64.49.13.27:4961 , piece_id=YFFLF6O7U4AGW3TAIQXLDGO4GITRL2PH5LK3WPQRJ5FIOK2UZTHA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.352363381 (4h55m44.939419603s) [noise=true , address=45.91.116.7:4031 , piece_id=RQO5CO4GIRLZH7ZV3PNBNA5CITIR4WPDBX2ANKC3QZIO4277NFAA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.352401102 (4h55m44.939457323s) [noise=true , address=104.36.82.58:4293 , piece_id=TJHAFPDX34JVXEKLLCXBUTJDULSCUB36XUMGQ77DWWMMLG6SDGGQ, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.352672198 (4h55m44.939728419s) [noise=true , address=104.36.82.55:4217 , piece_id=NVC2PGB6PAUW3FVTDK6QEGO5B5VF53HKFROP2XDOBFTZOYZJGOZQ, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.352686785 (4h55m44.939743006s) [noise=true , address=104.36.82.44:4901 , piece_id=4GQO4GJXY7V3XUHMH7HIG72NO43OEIVO5WZWJRDKOOHCOVU2UF2Q, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.352905193 (4h55m44.939961414s) [noise=true , address=104.36.82.35:4589 , piece_id=2TNQCRWILNH3LK5DL6KHOUWVRDD3VFQKTTJI43QZTEZ2H2QU6SUA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.352935029 (4h55m44.93999125s) [noise=true , address=104.36.82.15:4286 , piece_id=GAUDF7WSKVIEPTQAI6EGGIU336B2JRRJPNL6OY7KX3AFFDNKBBYA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.353130153 (4h55m44.940186374s) [noise=true , address=104.36.82.8:4987 , piece_id=UT4GNLD5ASIMDPTXPRDMY2PLQMTISUWWKKKD4Y5ZXZ4BQ44KHNIQ, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.353169336 (4h55m44.940225557s) [noise=true , address=104.36.82.16:4310 , piece_id=SBJICEMYSYAH53KD6MEYM7OCWSSWZYX46BSFADPNZZ4MPM6Z7HUQ, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.353422900 (4h55m44.940479121s) [noise=true , address=104.36.82.60:4367 , piece_id=MJA4TVZARNCFUTOSLH6KWSIP7ND4RKYD6OH7KYT3OFT37K2MANGA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.353656416 (4h55m44.940712637s) [noise=true , address=104.36.82.35:4592 , piece_id=YIERBYJGLQFGTRSOT4ZSJKO62MNDH3DUOAS65434RB2MSBX3EGOA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.353863271 (4h55m44.940919492s) [noise=true , address=104.36.82.17:4353 , piece_id=N5HNMPS45SEQRALZ7XT6FMHZFTS7FHFH24FXIOMVFC6VTSSRE2ZA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.354096647 (4h55m44.941152869s) [noise=true , address=104.36.82.5:4864 , piece_id=57RIVHQZ3KKNP5SWWVO4J6L5QGTU24P664SYEYF5BBYCA62Z2GPA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.354308964 (4h55m44.941365185s) [noise=true , address=104.36.82.56:4238 , piece_id=J5FYG5FAUX6WDIZDE6YQR6HHVTNREUCY3CCRQVG4AJIJO2ZY33BQ, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.354517353 (4h55m44.941573574s) [noise=true , address=104.36.82.12:4163 , piece_id=G3HPBKE766GXCXFCDJS36BCPGCHA4OVFK32VKVOV4NQGJKGIK67Q, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.355298460 (4h55m44.942354672s) [noise=true , address=104.36.82.37:4673 , piece_id=BSVONXWWBJHRDK7PZV7RYZMF4VCWG2LKKUFRBKZ6WYB4QLDN3ANA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.355833649 (4h55m44.942889861s) [noise=true , address=45.91.116.8:4065 , piece_id=RUAZW4OEX5ICVVC25Y2SXGXUMSYAYGGBASTKT35ISYCVCOP7BCZQ, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.356648310 (4h55m44.943704521s) [noise=true , address=79.127.201.215:4344 , piece_id=S7JKJHCXKZHEHF5YQA7F47A73ZLUXQRMJ3WGVXTQDSBFNO4OPMOA, seg_pos={0 0}, upload=5078]: Uploading piece... Aug 7 21:22:31.420527731 (4h55m45.007583942s) [upload=5078]: Next returned nil segment Aug 7 21:22:31.420540094 (4h55m45.007596305s) [upload=5078]: Waiting for error group managing segments... Aug 7 21:22:31.441095293 (4h55m45.028151514s) [noise=true , address=104.36.82.58:4293 , piece_id=TJHAFPDX34JVXEKLLCXBUTJDULSCUB36XUMGQ77DWWMMLG6SDGGQ, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.454894070 (4h55m45.041950301s) [noise=true , address=104.36.82.8:4987 , piece_id=UT4GNLD5ASIMDPTXPRDMY2PLQMTISUWWKKKD4Y5ZXZ4BQ44KHNIQ, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.466247081 (4h55m45.053303293s) [noise=true , address=104.36.82.35:4589 , piece_id=2TNQCRWILNH3LK5DL6KHOUWVRDD3VFQKTTJI43QZTEZ2H2QU6SUA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.467832441 (4h55m45.054888662s) [noise=true , address=45.91.116.8:4065 , piece_id=RUAZW4OEX5ICVVC25Y2SXGXUMSYAYGGBASTKT35ISYCVCOP7BCZQ, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.474570919 (4h55m45.06162715s) [noise=true , address=104.36.82.17:4353 , piece_id=N5HNMPS45SEQRALZ7XT6FMHZFTS7FHFH24FXIOMVFC6VTSSRE2ZA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.476328280 (4h55m45.063384491s) [noise=true , address=104.36.82.15:4286 , piece_id=GAUDF7WSKVIEPTQAI6EGGIU336B2JRRJPNL6OY7KX3AFFDNKBBYA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.476661793 (4h55m45.063718024s) [noise=true , address=104.36.82.60:4367 , piece_id=MJA4TVZARNCFUTOSLH6KWSIP7ND4RKYD6OH7KYT3OFT37K2MANGA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.486513301 (4h55m45.073569513s) [noise=true , address=104.36.82.56:4238 , piece_id=J5FYG5FAUX6WDIZDE6YQR6HHVTNREUCY3CCRQVG4AJIJO2ZY33BQ, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.487495355 (4h55m45.074551587s) [noise=true , address=104.36.82.55:4217 , piece_id=NVC2PGB6PAUW3FVTDK6QEGO5B5VF53HKFROP2XDOBFTZOYZJGOZQ, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.488952606 (4h55m45.076008827s) [noise=true , address=104.36.82.16:4310 , piece_id=SBJICEMYSYAH53KD6MEYM7OCWSSWZYX46BSFADPNZZ4MPM6Z7HUQ, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.492385343 (4h55m45.079441574s) [noise=true , address=104.36.82.35:4592 , piece_id=YIERBYJGLQFGTRSOT4ZSJKO62MNDH3DUOAS65434RB2MSBX3EGOA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.504789768 (4h55m45.091845999s) [noise=true , address=104.36.82.5:4864 , piece_id=57RIVHQZ3KKNP5SWWVO4J6L5QGTU24P664SYEYF5BBYCA62Z2GPA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.534017416 (4h55m45.121073647s) [noise=true , address=104.36.82.44:4901 , piece_id=4GQO4GJXY7V3XUHMH7HIG72NO43OEIVO5WZWJRDKOOHCOVU2UF2Q, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.594014387 (4h55m45.181070598s) [noise=true , address=79.127.201.215:4344 , piece_id=S7JKJHCXKZHEHF5YQA7F47A73ZLUXQRMJ3WGVXTQDSBFNO4OPMOA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: <nil> Aug 7 21:22:31.594041378 (4h55m45.181097599s) [seg_pos={0 0}, upload=5078]: Segment reached optimal threshold of 14 pieces. Aug 7 21:22:31.594296384 (4h55m45.181352596s) [noise=true , address=104.36.82.37:4673 , piece_id=BSVONXWWBJHRDK7PZV7RYZMF4VCWG2LKKUFRBKZ6WYB4QLDN3ANA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12NkBjmh5iL2BtckR19qgXQ4wnfvnbpA9erSuMCTEijwX7ZdHGL): protocol: write tcp 5.161.89.252:24122->104.36.82.37:4673: use of closed network connection; write tcp 5.161.89.252:24122->104.36.82.37:4673: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:24122->104.36.82.37:4673: use of closed network connection Aug 7 21:22:31.594425967 (4h55m45.181482207s) [noise=true , address=45.91.116.7:4031 , piece_id=RQO5CO4GIRLZH7ZV3PNBNA5CITIR4WPDBX2ANKC3QZIO4277NFAA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aAuYPWPUFSEEXPP9VMDZPDcT5qDJd7TKNDz9esCG7RpzF83GW): protocol: write tcp 5.161.89.252:42394->45.91.116.7:4031: use of closed network connection; write tcp 5.161.89.252:42394->45.91.116.7:4031: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:42394->45.91.116.7:4031: use of closed network connection Aug 7 21:22:31.594492841 (4h55m45.181549062s) [noise=true , address=64.49.13.27:4961 , piece_id=YFFLF6O7U4AGW3TAIQXLDGO4GITRL2PH5LK3WPQRJ5FIOK2UZTHA, seg_pos={0 0}, upload=5078]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwCHMdLXowSBhg3UtVahaWWAQ5TJYZ7yUSdAQ8Q8Cv3qWwMnt): protocol: read tcp 5.161.89.252:50748->64.49.13.27:4961: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:22:31.594530211 (4h55m45.181586432s) [noise=true , address=104.36.82.12:4163 , piece_id=G3HPBKE766GXCXFCDJS36BCPGCHA4OVFK32VKVOV4NQGJKGIK67Q, seg_pos={0 0}, upload=5078]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12w88UkGu5duYivJ6L6C9iJmTbnB63fDKiUUQprhwM3JshpXTdC): protocol: read tcp 5.161.89.252:58420->104.36.82.12:4163: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:22:31.594553194 (4h55m45.181609405s) [seg_pos={0 0}, upload=5078]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:22:31.594607895 (4h55m45.181664116s) [upload=5078]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:22:31.594611643 (4h55m45.181667864s) [upload=5078]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:31:08.619701228 (5h4m22.20675747s) [upload=5078]: Done uploading segments... Aug 7 21:31:08.619736956 (5h4m22.206793196s) [upload=5078]: Upload finished. err: metaclient: context deadline exceeded Duration: 9m24.674669272s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 1m0.001097123s FAIL (2025-08-07 21:31:05 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== nt: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:30.189934289 (4h54m43.776990511s) [upload=5071]: Starting upload Aug 7 21:21:30.189980476 (4h54m43.777036687s) [upload=5071]: Done starting upload Aug 7 21:21:30.190017424 (4h54m43.777073645s) [upload=5071]: Uploading segments... Aug 7 21:21:30.190043793 (4h54m43.777100005s) [upload=5071]: Got next segment. Inline: true Aug 7 21:21:30.190047100 (4h54m43.777103321s) [upload=5071]: Waiting for error group managing segments... Aug 7 21:21:30.190088458 (4h54m43.777144669s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:30.190092425 (4h54m43.777148646s) [upload=5071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:30.190094709 (4h54m43.77715093s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:30.179283539 (4h55m43.76633979s) [upload=5071]: Done uploading segments... Aug 7 21:22:30.179305229 (4h55m43.766361471s) [upload=5071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.041348848 (4h55m57.628405089s) [upload=5090]: Starting upload Aug 7 21:22:44.041428967 (4h55m57.628485208s) [upload=5090]: Done starting upload Aug 7 21:22:44.041486454 (4h55m57.628542695s) [upload=5090]: Uploading segments... Aug 7 21:22:44.041542930 (4h55m57.628599171s) [upload=5090]: Got next segment. Inline: true Aug 7 21:22:44.041549963 (4h55m57.628606204s) [upload=5090]: Waiting for error group managing segments... Aug 7 21:22:44.041596389 (4h55m57.628652631s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.041602170 (4h55m57.628658411s) [upload=5090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.041605667 (4h55m57.628661917s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.025768658 (4h56m57.612824879s) [upload=5090]: Done uploading segments... Aug 7 21:23:44.025784877 (4h56m57.612841089s) [upload=5090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:58.208173683 (4h57m11.795229924s) [upload=5112]: Starting upload Aug 7 21:23:58.208257669 (4h57m11.795313911s) [upload=5112]: Done starting upload Aug 7 21:23:58.208337199 (4h57m11.79539344s) [upload=5112]: Uploading segments... Aug 7 21:23:58.208378025 (4h57m11.795434265s) [upload=5112]: Got next segment. Inline: true Aug 7 21:23:58.208383675 (4h57m11.795439916s) [upload=5112]: Waiting for error group managing segments... Aug 7 21:23:58.208427437 (4h57m11.795483678s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:58.208433278 (4h57m11.795489519s) [upload=5112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:58.208438388 (4h57m11.795494628s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:58.195766004 (4h58m11.782822225s) [upload=5112]: Done uploading segments... Aug 7 21:24:58.195780731 (4h58m11.782836943s) [upload=5112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:04.854742735 (4h58m18.441798946s) [upload=5127]: Starting upload Aug 7 21:25:04.854807696 (4h58m18.441863907s) [upload=5127]: Done starting upload Aug 7 21:25:04.854870773 (4h58m18.441926984s) [upload=5127]: Uploading segments... Aug 7 21:25:04.854916199 (4h58m18.44197242s) [upload=5127]: Got next segment. Inline: true Aug 7 21:25:04.854936166 (4h58m18.441992387s) [upload=5127]: Waiting for error group managing segments... Aug 7 21:25:04.854972444 (4h58m18.442028655s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:04.854976711 (4h58m18.442032922s) [upload=5127]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:04.854979997 (4h58m18.442036218s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:04.841433540 (4h59m18.428489761s) [upload=5127]: Done uploading segments... Aug 7 21:26:04.841457825 (4h59m18.428514037s) [upload=5127]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:04.958090745 (4h59m18.545146957s) [upload=5139]: Starting upload Aug 7 21:26:04.958138415 (4h59m18.545194626s) [upload=5139]: Done starting upload Aug 7 21:26:04.958184982 (4h59m18.545241203s) [upload=5139]: Uploading segments... Aug 7 21:26:04.958220368 (4h59m18.545276589s) [upload=5139]: Got next segment. Inline: true Aug 7 21:26:04.958224295 (4h59m18.545280516s) [upload=5139]: Waiting for error group managing segments... Aug 7 21:26:04.958253820 (4h59m18.545310031s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:04.958257427 (4h59m18.545313638s) [upload=5139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:04.958259772 (4h59m18.545315993s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:04.939553376 (5h0m18.526609617s) [upload=5139]: Done uploading segments... Aug 7 21:27:04.939574906 (5h0m18.526631147s) [upload=5139]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:05.045401846 (5h0m18.632458067s) [upload=5157]: Starting upload Aug 7 21:27:05.045447422 (5h0m18.632503643s) [upload=5157]: Done starting upload Aug 7 21:27:05.045500470 (5h0m18.632556691s) [upload=5157]: Uploading segments... Aug 7 21:27:05.045530256 (5h0m18.632586467s) [upload=5157]: Got next segment. Inline: true Aug 7 21:27:05.045533973 (5h0m18.632590194s) [upload=5157]: Waiting for error group managing segments... Aug 7 21:27:05.045565873 (5h0m18.632622094s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:05.045571213 (5h0m18.632627434s) [upload=5157]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:05.045573487 (5h0m18.632629708s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:05.013376070 (5h1m18.600432281s) [upload=5157]: Done uploading segments... Aug 7 21:28:05.013391970 (5h1m18.600448181s) [upload=5157]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:05.107129855 (5h1m18.694186067s) [upload=5174]: Starting upload Aug 7 21:28:05.107184377 (5h1m18.694240598s) [upload=5174]: Done starting upload Aug 7 21:28:05.107240842 (5h1m18.694297063s) [upload=5174]: Uploading segments... Aug 7 21:28:05.107277301 (5h1m18.694333512s) [upload=5174]: Got next segment. Inline: true Aug 7 21:28:05.107280647 (5h1m18.694336868s) [upload=5174]: Waiting for error group managing segments... Aug 7 21:28:05.107312757 (5h1m18.694368969s) [upload=5174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:05.107316373 (5h1m18.694372584s) [upload=5174]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:05.107318867 (5h1m18.694375088s) [upload=5174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:05.085592777 (5h2m18.672648989s) [upload=5174]: Done uploading segments... Aug 7 21:29:05.085611523 (5h2m18.672667733s) [upload=5174]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:05.172798369 (5h2m18.75985459s) [upload=5191]: Starting upload Aug 7 21:29:05.172850326 (5h2m18.759906537s) [upload=5191]: Done starting upload Aug 7 21:29:05.172893617 (5h2m18.759949839s) [upload=5191]: Uploading segments... Aug 7 21:29:05.172921278 (5h2m18.759977499s) [upload=5191]: Got next segment. Inline: true Aug 7 21:29:05.172924825 (5h2m18.759981046s) [upload=5191]: Waiting for error group managing segments... Aug 7 21:29:05.172954671 (5h2m18.760010882s) [upload=5191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:05.172958317 (5h2m18.760014538s) [upload=5191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:05.172960632 (5h2m18.760016843s) [upload=5191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:05.162199693 (5h3m18.749255944s) [upload=5191]: Done uploading segments... Aug 7 21:30:05.162220462 (5h3m18.749276703s) [upload=5191]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:30:05.259129610 (5h3m18.846185831s) [upload=5205]: Starting upload Aug 7 21:30:05.259178532 (5h3m18.846234742s) [upload=5205]: Done starting upload Aug 7 21:30:05.259225909 (5h3m18.846282131s) [upload=5205]: Uploading segments... Aug 7 21:30:05.259254723 (5h3m18.846310935s) [upload=5205]: Got next segment. Inline: true Aug 7 21:30:05.259258370 (5h3m18.846314591s) [upload=5205]: Waiting for error group managing segments... Aug 7 21:30:05.259289909 (5h3m18.846346131s) [upload=5205]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:05.259293515 (5h3m18.846349736s) [upload=5205]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:30:05.259295659 (5h3m18.84635187s) [upload=5205]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:31:05.245621000 (5h4m18.83267725s) [upload=5205]: Done uploading segments... Aug 7 21:31:05.245643692 (5h4m18.832699933s) [upload=5205]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.00111673s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 25 out of 60 is failed (threshold=3) [.................................XX..XXXXXXXXXXXXXXXXXXXXXXX] duration: 30.001473782s FAIL (2025-08-07 21:31:01 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001487788s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 172.567173ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:31:00 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 172.592359ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 30.000399406s FAIL (2025-08-07 21:30:53 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000412781s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration: 30.000338211s FAIL (2025-08-07 21:30:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000415875s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 30.000828775s FAIL (2025-08-07 21:30:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001106122s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.000810989s FAIL (2025-08-07 21:30:37 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== d=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Aug 7 21:22:41.968187982 (4h55m55.555244194s) [upload=5089]: Starting upload Aug 7 21:22:41.968245641 (4h55m55.555301862s) [upload=5089]: Done starting upload Aug 7 21:22:41.968339846 (4h55m55.555396087s) [upload=5089]: Uploading segments... Aug 7 21:22:41.968596806 (4h55m55.555653047s) [upload=5089]: Got next segment. Inline: false Aug 7 21:22:41.968638122 (4h55m55.555694354s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Aug 7 21:23:20.066472018 (4h56m33.653528259s) [upload=5100]: Starting upload Aug 7 21:23:20.066538723 (4h56m33.653594965s) [upload=5100]: Done starting upload Aug 7 21:23:20.066650672 (4h56m33.653706913s) [upload=5100]: Uploading segments... Aug 7 21:23:20.066882696 (4h56m33.653938936s) [upload=5100]: Got next segment. Inline: false Aug 7 21:23:20.066929522 (4h56m33.653985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Aug 7 21:23:50.132503139 (4h57m3.719559361s) [upload=5111]: Starting upload Aug 7 21:23:50.132542101 (4h57m3.719598322s) [upload=5111]: Done starting upload Aug 7 21:23:50.132574512 (4h57m3.719630733s) [upload=5111]: Uploading segments... Aug 7 21:23:50.132772312 (4h57m3.719828533s) [upload=5111]: Got next segment. Inline: false Aug 7 21:23:50.132810694 (4h57m3.719866904s) [upload=5111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:50.132814390 (4h57m3.719870611s) [upload=5111]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:20.119434924 (4h57m33.706491135s) [upload=5111]: Next segment err: context deadline exceeded Aug 7 21:24:20.119485478 (4h57m33.706541699s) [upload=5111]: Done uploading segments... Aug 7 21:24:20.119497891 (4h57m33.706554102s) [upload=5111]: Upload finished. err: context deadline exceeded Aug 7 21:24:45.494243280 (4h57m59.081299501s) [upload=5120]: Starting upload Aug 7 21:24:45.494313381 (4h57m59.081369612s) [upload=5120]: Done starting upload Aug 7 21:24:45.494474943 (4h57m59.081531184s) [upload=5120]: Uploading segments... Aug 7 21:24:45.494627989 (4h57m59.08168422s) [upload=5120]: Got next segment. Inline: false Aug 7 21:24:45.494663625 (4h57m59.081719856s) [upload=5120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:45.494668073 (4h57m59.081724294s) [upload=5120]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:15.480538548 (4h58m29.067594769s) [upload=5120]: Next segment err: context deadline exceeded Aug 7 21:25:15.480665896 (4h58m29.067722117s) [upload=5120]: Done uploading segments... Aug 7 21:25:15.480717081 (4h58m29.067773302s) [upload=5120]: Upload finished. err: context deadline exceeded Aug 7 21:25:15.571225478 (4h58m29.158281699s) [upload=5129]: Starting upload Aug 7 21:25:15.571281894 (4h58m29.158338105s) [upload=5129]: Done starting upload Aug 7 21:25:15.571359468 (4h58m29.158415689s) [upload=5129]: Uploading segments... Aug 7 21:25:15.571564180 (4h58m29.158620401s) [upload=5129]: Got next segment. Inline: false Aug 7 21:25:15.571622810 (4h58m29.158679021s) [upload=5129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.571628479 (4h58m29.1586847s) [upload=5129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:45.547010083 (4h58m59.134066294s) [upload=5129]: Next segment err: context deadline exceeded Aug 7 21:25:45.547084462 (4h58m59.134140683s) [upload=5129]: Done uploading segments... Aug 7 21:25:45.547107044 (4h58m59.134163266s) [upload=5129]: Upload finished. err: context deadline exceeded Aug 7 21:25:45.641521062 (4h58m59.228577283s) [upload=5137]: Starting upload Aug 7 21:25:45.641578831 (4h58m59.228635041s) [upload=5137]: Done starting upload Aug 7 21:25:45.641631238 (4h58m59.228687459s) [upload=5137]: Uploading segments... Aug 7 21:25:45.641794333 (4h58m59.228850554s) [upload=5137]: Got next segment. Inline: false Aug 7 21:25:45.641818608 (4h58m59.228874829s) [upload=5137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.641822736 (4h58m59.228878947s) [upload=5137]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:15.630688562 (4h59m29.217744793s) [upload=5137]: Next segment err: context deadline exceeded Aug 7 21:26:15.630873647 (4h59m29.217929858s) [upload=5137]: Done uploading segments... Aug 7 21:26:15.630892703 (4h59m29.217948914s) [upload=5137]: Upload finished. err: context deadline exceeded Aug 7 21:26:29.805240268 (4h59m43.392296479s) [upload=5148]: Starting upload Aug 7 21:26:29.805289190 (4h59m43.392345411s) [upload=5148]: Done starting upload Aug 7 21:26:29.805393004 (4h59m43.392449225s) [upload=5148]: Uploading segments... Aug 7 21:26:29.805516605 (4h59m43.392572816s) [upload=5148]: Got next segment. Inline: false Aug 7 21:26:29.805549756 (4h59m43.392605967s) [upload=5148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.805553633 (4h59m43.392609854s) [upload=5148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:59.790298417 (5h0m13.377354628s) [upload=5148]: Next segment err: context deadline exceeded Aug 7 21:26:59.790450801 (5h0m13.377507012s) [upload=5148]: Done uploading segments... Aug 7 21:26:59.790479374 (5h0m13.377535595s) [upload=5148]: Upload finished. err: context deadline exceeded Aug 7 21:26:59.854535904 (5h0m13.441592145s) [upload=5156]: Starting upload Aug 7 21:26:59.854627225 (5h0m13.441683465s) [upload=5156]: Done starting upload Aug 7 21:26:59.854723153 (5h0m13.441779394s) [upload=5156]: Uploading segments... Aug 7 21:26:59.855007455 (5h0m13.442063705s) [upload=5156]: Got next segment. Inline: false Aug 7 21:26:59.855059541 (5h0m13.442115782s) [upload=5156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:59.855096871 (5h0m13.442153123s) [upload=5156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:27:29.841757946 (5h0m43.428814187s) [upload=5156]: Next segment err: context deadline exceeded Aug 7 21:27:29.841960274 (5h0m43.429016514s) [upload=5156]: Done uploading segments... Aug 7 21:27:29.842007351 (5h0m43.429063593s) [upload=5156]: Upload finished. err: context deadline exceeded Aug 7 21:28:06.370955849 (5h1m19.95801209s) [upload=5175]: Starting upload Aug 7 21:28:06.371030067 (5h1m19.958086308s) [upload=5175]: Done starting upload Aug 7 21:28:06.371151734 (5h1m19.958207975s) [upload=5175]: Uploading segments... Aug 7 21:28:06.371451254 (5h1m19.958507495s) [upload=5175]: Got next segment. Inline: false Aug 7 21:28:06.371506767 (5h1m19.958563008s) [upload=5175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:06.371512959 (5h1m19.958569209s) [upload=5175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:28:36.358597390 (5h1m49.945653632s) [upload=5175]: Next segment err: context deadline exceeded Aug 7 21:28:36.358673913 (5h1m49.945730154s) [upload=5175]: Done uploading segments... Aug 7 21:28:36.358704921 (5h1m49.945761162s) [upload=5175]: Upload finished. err: context deadline exceeded Aug 7 21:29:01.102431480 (5h2m14.689487702s) [upload=5189]: Starting upload Aug 7 21:29:01.102546615 (5h2m14.689602836s) [upload=5189]: Done starting upload Aug 7 21:29:01.102685454 (5h2m14.689741675s) [upload=5189]: Uploading segments... Aug 7 21:29:01.102866763 (5h2m14.689922984s) [upload=5189]: Got next segment. Inline: false Aug 7 21:29:01.102903291 (5h2m14.689959502s) [upload=5189]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:01.102907839 (5h2m14.68996406s) [upload=5189]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:29:31.087349640 (5h2m44.67440585s) [upload=5189]: Next segment err: context deadline exceeded Aug 7 21:29:31.087429919 (5h2m44.67448613s) [upload=5189]: Done uploading segments... Aug 7 21:29:31.087444957 (5h2m44.674501178s) [upload=5189]: Upload finished. err: context deadline exceeded Aug 7 21:30:07.260405000 (5h3m20.847461241s) [upload=5206]: Starting upload Aug 7 21:30:07.260495098 (5h3m20.84755135s) [upload=5206]: Done starting upload Aug 7 21:30:07.260642223 (5h3m20.847698474s) [upload=5206]: Uploading segments... Aug 7 21:30:07.260927024 (5h3m20.847983276s) [upload=5206]: Got next segment. Inline: false Aug 7 21:30:07.260984843 (5h3m20.848041084s) [upload=5206]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:30:07.260994751 (5h3m20.848050993s) [upload=5206]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:30:37.232137150 (5h3m50.81919336s) [upload=5206]: Next segment err: context deadline exceeded Aug 7 21:30:37.232307578 (5h3m50.819363789s) [upload=5206]: Done uploading segments... Aug 7 21:30:37.232355838 (5h3m50.819412059s) [upload=5206]: Upload finished. err: context deadline exceeded Duration: 30.000834194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 987.972843ms FAIL (2025-08-07 21:30:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 987.992802ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 24 out of 60 is failed (threshold=3) [..................................XX..XXXXXXXXXXXXXXXXXXXXXX] duration: 30.000464144s FAIL (2025-08-07 21:30:31 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000476508s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [..............................................X........XX...] duration: 221.501103ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:30:22 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 221.522864ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X........XX..] duration: 1m52.346129899s remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 21:30:21 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 1m52.346153463s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 17 out of 60 is failed (threshold=3) [..........................................XX.XXXXXXXXXXXXXXX] duration: 1m0.00036917s FAIL (2025-08-07 21:30:18 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000381714s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.001347068s FAIL (2025-08-07 21:30:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001365823s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 30.001095458s FAIL (2025-08-07 21:30:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001112289s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 116.849587ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 21:30:06 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 116.868853ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 1m0.00145819s FAIL (2025-08-07 21:30:05 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== t deadline exceeded Aug 7 21:19:30.042411605 (4h52m43.629467826s) [upload=5040]: Starting upload Aug 7 21:19:30.042491373 (4h52m43.629547594s) [upload=5040]: Done starting upload Aug 7 21:19:30.042544984 (4h52m43.629601195s) [upload=5040]: Uploading segments... Aug 7 21:19:30.042585870 (4h52m43.629642091s) [upload=5040]: Got next segment. Inline: true Aug 7 21:19:30.042590809 (4h52m43.62964703s) [upload=5040]: Waiting for error group managing segments... Aug 7 21:19:30.042637226 (4h52m43.629693436s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:30.042642325 (4h52m43.629698546s) [upload=5040]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:30.042647144 (4h52m43.629703365s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:30.020861737 (4h53m43.607917958s) [upload=5040]: Done uploading segments... Aug 7 21:20:30.020878959 (4h53m43.60793518s) [upload=5040]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:30.189934289 (4h54m43.776990511s) [upload=5071]: Starting upload Aug 7 21:21:30.189980476 (4h54m43.777036687s) [upload=5071]: Done starting upload Aug 7 21:21:30.190017424 (4h54m43.777073645s) [upload=5071]: Uploading segments... Aug 7 21:21:30.190043793 (4h54m43.777100005s) [upload=5071]: Got next segment. Inline: true Aug 7 21:21:30.190047100 (4h54m43.777103321s) [upload=5071]: Waiting for error group managing segments... Aug 7 21:21:30.190088458 (4h54m43.777144669s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:30.190092425 (4h54m43.777148646s) [upload=5071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:30.190094709 (4h54m43.77715093s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:30.179283539 (4h55m43.76633979s) [upload=5071]: Done uploading segments... Aug 7 21:22:30.179305229 (4h55m43.766361471s) [upload=5071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.041348848 (4h55m57.628405089s) [upload=5090]: Starting upload Aug 7 21:22:44.041428967 (4h55m57.628485208s) [upload=5090]: Done starting upload Aug 7 21:22:44.041486454 (4h55m57.628542695s) [upload=5090]: Uploading segments... Aug 7 21:22:44.041542930 (4h55m57.628599171s) [upload=5090]: Got next segment. Inline: true Aug 7 21:22:44.041549963 (4h55m57.628606204s) [upload=5090]: Waiting for error group managing segments... Aug 7 21:22:44.041596389 (4h55m57.628652631s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.041602170 (4h55m57.628658411s) [upload=5090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.041605667 (4h55m57.628661917s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.025768658 (4h56m57.612824879s) [upload=5090]: Done uploading segments... Aug 7 21:23:44.025784877 (4h56m57.612841089s) [upload=5090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:58.208173683 (4h57m11.795229924s) [upload=5112]: Starting upload Aug 7 21:23:58.208257669 (4h57m11.795313911s) [upload=5112]: Done starting upload Aug 7 21:23:58.208337199 (4h57m11.79539344s) [upload=5112]: Uploading segments... Aug 7 21:23:58.208378025 (4h57m11.795434265s) [upload=5112]: Got next segment. Inline: true Aug 7 21:23:58.208383675 (4h57m11.795439916s) [upload=5112]: Waiting for error group managing segments... Aug 7 21:23:58.208427437 (4h57m11.795483678s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:58.208433278 (4h57m11.795489519s) [upload=5112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:58.208438388 (4h57m11.795494628s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:58.195766004 (4h58m11.782822225s) [upload=5112]: Done uploading segments... Aug 7 21:24:58.195780731 (4h58m11.782836943s) [upload=5112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:04.854742735 (4h58m18.441798946s) [upload=5127]: Starting upload Aug 7 21:25:04.854807696 (4h58m18.441863907s) [upload=5127]: Done starting upload Aug 7 21:25:04.854870773 (4h58m18.441926984s) [upload=5127]: Uploading segments... Aug 7 21:25:04.854916199 (4h58m18.44197242s) [upload=5127]: Got next segment. Inline: true Aug 7 21:25:04.854936166 (4h58m18.441992387s) [upload=5127]: Waiting for error group managing segments... Aug 7 21:25:04.854972444 (4h58m18.442028655s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:04.854976711 (4h58m18.442032922s) [upload=5127]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:04.854979997 (4h58m18.442036218s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:04.841433540 (4h59m18.428489761s) [upload=5127]: Done uploading segments... Aug 7 21:26:04.841457825 (4h59m18.428514037s) [upload=5127]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:04.958090745 (4h59m18.545146957s) [upload=5139]: Starting upload Aug 7 21:26:04.958138415 (4h59m18.545194626s) [upload=5139]: Done starting upload Aug 7 21:26:04.958184982 (4h59m18.545241203s) [upload=5139]: Uploading segments... Aug 7 21:26:04.958220368 (4h59m18.545276589s) [upload=5139]: Got next segment. Inline: true Aug 7 21:26:04.958224295 (4h59m18.545280516s) [upload=5139]: Waiting for error group managing segments... Aug 7 21:26:04.958253820 (4h59m18.545310031s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:04.958257427 (4h59m18.545313638s) [upload=5139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:04.958259772 (4h59m18.545315993s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:04.939553376 (5h0m18.526609617s) [upload=5139]: Done uploading segments... Aug 7 21:27:04.939574906 (5h0m18.526631147s) [upload=5139]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:05.045401846 (5h0m18.632458067s) [upload=5157]: Starting upload Aug 7 21:27:05.045447422 (5h0m18.632503643s) [upload=5157]: Done starting upload Aug 7 21:27:05.045500470 (5h0m18.632556691s) [upload=5157]: Uploading segments... Aug 7 21:27:05.045530256 (5h0m18.632586467s) [upload=5157]: Got next segment. Inline: true Aug 7 21:27:05.045533973 (5h0m18.632590194s) [upload=5157]: Waiting for error group managing segments... Aug 7 21:27:05.045565873 (5h0m18.632622094s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:05.045571213 (5h0m18.632627434s) [upload=5157]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:05.045573487 (5h0m18.632629708s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:05.013376070 (5h1m18.600432281s) [upload=5157]: Done uploading segments... Aug 7 21:28:05.013391970 (5h1m18.600448181s) [upload=5157]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:05.107129855 (5h1m18.694186067s) [upload=5174]: Starting upload Aug 7 21:28:05.107184377 (5h1m18.694240598s) [upload=5174]: Done starting upload Aug 7 21:28:05.107240842 (5h1m18.694297063s) [upload=5174]: Uploading segments... Aug 7 21:28:05.107277301 (5h1m18.694333512s) [upload=5174]: Got next segment. Inline: true Aug 7 21:28:05.107280647 (5h1m18.694336868s) [upload=5174]: Waiting for error group managing segments... Aug 7 21:28:05.107312757 (5h1m18.694368969s) [upload=5174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:05.107316373 (5h1m18.694372584s) [upload=5174]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:05.107318867 (5h1m18.694375088s) [upload=5174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:05.085592777 (5h2m18.672648989s) [upload=5174]: Done uploading segments... Aug 7 21:29:05.085611523 (5h2m18.672667733s) [upload=5174]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:05.172798369 (5h2m18.75985459s) [upload=5191]: Starting upload Aug 7 21:29:05.172850326 (5h2m18.759906537s) [upload=5191]: Done starting upload Aug 7 21:29:05.172893617 (5h2m18.759949839s) [upload=5191]: Uploading segments... Aug 7 21:29:05.172921278 (5h2m18.759977499s) [upload=5191]: Got next segment. Inline: true Aug 7 21:29:05.172924825 (5h2m18.759981046s) [upload=5191]: Waiting for error group managing segments... Aug 7 21:29:05.172954671 (5h2m18.760010882s) [upload=5191]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:05.172958317 (5h2m18.760014538s) [upload=5191]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:05.172960632 (5h2m18.760016843s) [upload=5191]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:05.162199693 (5h3m18.749255944s) [upload=5191]: Done uploading segments... Aug 7 21:30:05.162220462 (5h3m18.749276703s) [upload=5191]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001473047s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 23 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration: 30.000547283s FAIL (2025-08-07 21:30:03 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== : metaclient: context deadline exceeded Aug 7 21:23:14.781991861 (4h56m28.369048072s) [upload=5099]: Starting upload Aug 7 21:23:14.782035392 (4h56m28.369091604s) [upload=5099]: Done starting upload Aug 7 21:23:14.782087559 (4h56m28.369143771s) [upload=5099]: Uploading segments... Aug 7 21:23:14.782124759 (4h56m28.369180991s) [upload=5099]: Got next segment. Inline: true Aug 7 21:23:14.782130029 (4h56m28.369186241s) [upload=5099]: Waiting for error group managing segments... Aug 7 21:23:14.782161368 (4h56m28.369217579s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.782164914 (4h56m28.369221126s) [upload=5099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.782167850 (4h56m28.369224061s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.771505754 (4h56m58.358561975s) [upload=5099]: Done uploading segments... Aug 7 21:23:44.771536371 (4h56m58.358592583s) [upload=5099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:44.852200021 (4h56m58.439256242s) [upload=5107]: Starting upload Aug 7 21:23:44.852251448 (4h56m58.439307669s) [upload=5107]: Done starting upload Aug 7 21:23:44.852291703 (4h56m58.439347924s) [upload=5107]: Uploading segments... Aug 7 21:23:44.852320757 (4h56m58.439376978s) [upload=5107]: Got next segment. Inline: true Aug 7 21:23:44.852324224 (4h56m58.439380445s) [upload=5107]: Waiting for error group managing segments... Aug 7 21:23:44.852355542 (4h56m58.439411753s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:44.852359279 (4h56m58.4394155s) [upload=5107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:44.852361614 (4h56m58.439417824s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.835050939 (4h57m28.42210715s) [upload=5107]: Done uploading segments... Aug 7 21:24:14.835115480 (4h57m28.422171701s) [upload=5107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:56.715637130 (4h58m10.302693351s) [upload=5124]: Starting upload Aug 7 21:24:56.715691522 (4h58m10.302747743s) [upload=5124]: Done starting upload Aug 7 21:24:56.715738660 (4h58m10.302794881s) [upload=5124]: Uploading segments... Aug 7 21:24:56.715821674 (4h58m10.302877895s) [upload=5124]: Got next segment. Inline: true Aug 7 21:24:56.715825752 (4h58m10.302881973s) [upload=5124]: Waiting for error group managing segments... Aug 7 21:24:56.715858083 (4h58m10.302914294s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:56.715861810 (4h58m10.302918031s) [upload=5124]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:56.715864014 (4h58m10.302920235s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:26.698879559 (4h58m40.285935771s) [upload=5124]: Done uploading segments... Aug 7 21:25:26.698896120 (4h58m40.285952331s) [upload=5124]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:27.243732093 (4h58m40.830788304s) [upload=5132]: Starting upload Aug 7 21:25:27.243787377 (4h58m40.830843598s) [upload=5132]: Done starting upload Aug 7 21:25:27.243846396 (4h58m40.830902607s) [upload=5132]: Uploading segments... Aug 7 21:25:27.243883505 (4h58m40.830939727s) [upload=5132]: Got next segment. Inline: true Aug 7 21:25:27.243887303 (4h58m40.830943524s) [upload=5132]: Waiting for error group managing segments... Aug 7 21:25:27.243942507 (4h58m40.830998728s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:27.243946624 (4h58m40.831002845s) [upload=5132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:27.243948898 (4h58m40.831005119s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.223887428 (4h59m10.810943669s) [upload=5132]: Done uploading segments... Aug 7 21:25:57.223909498 (4h59m10.81096574s) [upload=5132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:29.142150756 (4h59m42.729206977s) [upload=5147]: Starting upload Aug 7 21:26:29.142194398 (4h59m42.729250619s) [upload=5147]: Done starting upload Aug 7 21:26:29.142233742 (4h59m42.729289963s) [upload=5147]: Uploading segments... Aug 7 21:26:29.142260401 (4h59m42.729316612s) [upload=5147]: Got next segment. Inline: true Aug 7 21:26:29.142263868 (4h59m42.729320089s) [upload=5147]: Waiting for error group managing segments... Aug 7 21:26:29.142293002 (4h59m42.729349213s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.142297080 (4h59m42.729353301s) [upload=5147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:29.142299354 (4h59m42.729355575s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:59.127042625 (5h0m12.714098866s) [upload=5147]: Done uploading segments... Aug 7 21:26:59.127117424 (5h0m12.714173665s) [upload=5147]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:10.814290180 (5h0m24.401346421s) [upload=5160]: Starting upload Aug 7 21:27:10.814371522 (5h0m24.401427764s) [upload=5160]: Done starting upload Aug 7 21:27:10.814438597 (5h0m24.401494839s) [upload=5160]: Uploading segments... Aug 7 21:27:10.814480476 (5h0m24.401536717s) [upload=5160]: Got next segment. Inline: true Aug 7 21:27:10.814485826 (5h0m24.401542067s) [upload=5160]: Waiting for error group managing segments... Aug 7 21:27:10.814529677 (5h0m24.401585918s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:10.814535198 (5h0m24.401591438s) [upload=5160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:10.814538994 (5h0m24.401595236s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:40.714926118 (5h0m54.301982339s) [upload=5160]: Done uploading segments... Aug 7 21:27:40.714942067 (5h0m54.301998278s) [upload=5160]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:40.796846333 (5h0m54.383902554s) [upload=5169]: Starting upload Aug 7 21:27:40.796900224 (5h0m54.383956435s) [upload=5169]: Done starting upload Aug 7 21:27:40.796945107 (5h0m54.384001318s) [upload=5169]: Uploading segments... Aug 7 21:27:40.796979782 (5h0m54.384036003s) [upload=5169]: Got next segment. Inline: true Aug 7 21:27:40.796983369 (5h0m54.38403959s) [upload=5169]: Waiting for error group managing segments... Aug 7 21:27:40.797013695 (5h0m54.384069907s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:40.797017642 (5h0m54.384073863s) [upload=5169]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:40.797020007 (5h0m54.384076218s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:10.771421400 (5h1m24.358477651s) [upload=5169]: Done uploading segments... Aug 7 21:28:10.771479779 (5h1m24.35853602s) [upload=5169]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:23.151092147 (5h1m36.738148388s) [upload=5182]: Starting upload Aug 7 21:28:23.151190951 (5h1m36.738247193s) [upload=5182]: Done starting upload Aug 7 21:28:23.151342555 (5h1m36.738398796s) [upload=5182]: Uploading segments... Aug 7 21:28:23.151386697 (5h1m36.738442937s) [upload=5182]: Got next segment. Inline: true Aug 7 21:28:23.151392588 (5h1m36.738448829s) [upload=5182]: Waiting for error group managing segments... Aug 7 21:28:23.151436370 (5h1m36.738492611s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:23.151441740 (5h1m36.738497981s) [upload=5182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:23.151445377 (5h1m36.738501618s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:53.132183214 (5h2m6.719239455s) [upload=5182]: Done uploading segments... Aug 7 21:28:53.132205376 (5h2m6.719261617s) [upload=5182]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:03.059565675 (5h2m16.646621886s) [upload=5190]: Starting upload Aug 7 21:29:03.059620448 (5h2m16.646676659s) [upload=5190]: Done starting upload Aug 7 21:29:03.059662505 (5h2m16.646718727s) [upload=5190]: Uploading segments... Aug 7 21:29:03.059702511 (5h2m16.646758732s) [upload=5190]: Got next segment. Inline: true Aug 7 21:29:03.059707350 (5h2m16.646763561s) [upload=5190]: Waiting for error group managing segments... Aug 7 21:29:03.059743357 (5h2m16.646799578s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:03.059748677 (5h2m16.646804898s) [upload=5190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:03.059752263 (5h2m16.646808474s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:33.039700593 (5h2m46.626756834s) [upload=5190]: Done uploading segments... Aug 7 21:29:33.039722394 (5h2m46.626778635s) [upload=5190]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:33.107560385 (5h2m46.694616626s) [upload=5202]: Starting upload Aug 7 21:29:33.107638791 (5h2m46.694695032s) [upload=5202]: Done starting upload Aug 7 21:29:33.107697620 (5h2m46.694753871s) [upload=5202]: Uploading segments... Aug 7 21:29:33.107747444 (5h2m46.694803705s) [upload=5202]: Got next segment. Inline: true Aug 7 21:29:33.107753225 (5h2m46.694809465s) [upload=5202]: Waiting for error group managing segments... Aug 7 21:29:33.107796575 (5h2m46.694852816s) [upload=5202]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:33.107802446 (5h2m46.694858697s) [upload=5202]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:33.107805812 (5h2m46.694862053s) [upload=5202]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:30:03.094304693 (5h3m16.681360904s) [upload=5202]: Done uploading segments... Aug 7 21:30:03.094320953 (5h3m16.681377174s) [upload=5202]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000559526s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 23 out of 60 is failed (threshold=3) [...................................XX..XXXXXXXXXXXXXXXXXXXXX] duration: 30.001408368s FAIL (2025-08-07 21:30:01 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001422886s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 30.001306209s FAIL (2025-08-07 21:29:51 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001326757s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 30.001198545s FAIL (2025-08-07 21:29:33 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ent: context deadline exceeded Aug 7 21:22:44.704346518 (4h55m58.291402739s) [upload=5092]: Starting upload Aug 7 21:22:44.704395569 (4h55m58.29145179s) [upload=5092]: Done starting upload Aug 7 21:22:44.704432468 (4h55m58.291488689s) [upload=5092]: Uploading segments... Aug 7 21:22:44.704457685 (4h55m58.291513906s) [upload=5092]: Got next segment. Inline: true Aug 7 21:22:44.704461182 (4h55m58.291517403s) [upload=5092]: Waiting for error group managing segments... Aug 7 21:22:44.704487161 (4h55m58.291543382s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.704490647 (4h55m58.291546868s) [upload=5092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.704493612 (4h55m58.291549833s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.694775680 (4h56m28.281831901s) [upload=5092]: Done uploading segments... Aug 7 21:23:14.694790848 (4h56m28.281847069s) [upload=5092]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.781991861 (4h56m28.369048072s) [upload=5099]: Starting upload Aug 7 21:23:14.782035392 (4h56m28.369091604s) [upload=5099]: Done starting upload Aug 7 21:23:14.782087559 (4h56m28.369143771s) [upload=5099]: Uploading segments... Aug 7 21:23:14.782124759 (4h56m28.369180991s) [upload=5099]: Got next segment. Inline: true Aug 7 21:23:14.782130029 (4h56m28.369186241s) [upload=5099]: Waiting for error group managing segments... Aug 7 21:23:14.782161368 (4h56m28.369217579s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.782164914 (4h56m28.369221126s) [upload=5099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.782167850 (4h56m28.369224061s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.771505754 (4h56m58.358561975s) [upload=5099]: Done uploading segments... Aug 7 21:23:44.771536371 (4h56m58.358592583s) [upload=5099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:44.852200021 (4h56m58.439256242s) [upload=5107]: Starting upload Aug 7 21:23:44.852251448 (4h56m58.439307669s) [upload=5107]: Done starting upload Aug 7 21:23:44.852291703 (4h56m58.439347924s) [upload=5107]: Uploading segments... Aug 7 21:23:44.852320757 (4h56m58.439376978s) [upload=5107]: Got next segment. Inline: true Aug 7 21:23:44.852324224 (4h56m58.439380445s) [upload=5107]: Waiting for error group managing segments... Aug 7 21:23:44.852355542 (4h56m58.439411753s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:44.852359279 (4h56m58.4394155s) [upload=5107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:44.852361614 (4h56m58.439417824s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.835050939 (4h57m28.42210715s) [upload=5107]: Done uploading segments... Aug 7 21:24:14.835115480 (4h57m28.422171701s) [upload=5107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:56.715637130 (4h58m10.302693351s) [upload=5124]: Starting upload Aug 7 21:24:56.715691522 (4h58m10.302747743s) [upload=5124]: Done starting upload Aug 7 21:24:56.715738660 (4h58m10.302794881s) [upload=5124]: Uploading segments... Aug 7 21:24:56.715821674 (4h58m10.302877895s) [upload=5124]: Got next segment. Inline: true Aug 7 21:24:56.715825752 (4h58m10.302881973s) [upload=5124]: Waiting for error group managing segments... Aug 7 21:24:56.715858083 (4h58m10.302914294s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:56.715861810 (4h58m10.302918031s) [upload=5124]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:56.715864014 (4h58m10.302920235s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:26.698879559 (4h58m40.285935771s) [upload=5124]: Done uploading segments... Aug 7 21:25:26.698896120 (4h58m40.285952331s) [upload=5124]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:27.243732093 (4h58m40.830788304s) [upload=5132]: Starting upload Aug 7 21:25:27.243787377 (4h58m40.830843598s) [upload=5132]: Done starting upload Aug 7 21:25:27.243846396 (4h58m40.830902607s) [upload=5132]: Uploading segments... Aug 7 21:25:27.243883505 (4h58m40.830939727s) [upload=5132]: Got next segment. Inline: true Aug 7 21:25:27.243887303 (4h58m40.830943524s) [upload=5132]: Waiting for error group managing segments... Aug 7 21:25:27.243942507 (4h58m40.830998728s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:27.243946624 (4h58m40.831002845s) [upload=5132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:27.243948898 (4h58m40.831005119s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.223887428 (4h59m10.810943669s) [upload=5132]: Done uploading segments... Aug 7 21:25:57.223909498 (4h59m10.81096574s) [upload=5132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:29.142150756 (4h59m42.729206977s) [upload=5147]: Starting upload Aug 7 21:26:29.142194398 (4h59m42.729250619s) [upload=5147]: Done starting upload Aug 7 21:26:29.142233742 (4h59m42.729289963s) [upload=5147]: Uploading segments... Aug 7 21:26:29.142260401 (4h59m42.729316612s) [upload=5147]: Got next segment. Inline: true Aug 7 21:26:29.142263868 (4h59m42.729320089s) [upload=5147]: Waiting for error group managing segments... Aug 7 21:26:29.142293002 (4h59m42.729349213s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.142297080 (4h59m42.729353301s) [upload=5147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:29.142299354 (4h59m42.729355575s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:59.127042625 (5h0m12.714098866s) [upload=5147]: Done uploading segments... Aug 7 21:26:59.127117424 (5h0m12.714173665s) [upload=5147]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:10.814290180 (5h0m24.401346421s) [upload=5160]: Starting upload Aug 7 21:27:10.814371522 (5h0m24.401427764s) [upload=5160]: Done starting upload Aug 7 21:27:10.814438597 (5h0m24.401494839s) [upload=5160]: Uploading segments... Aug 7 21:27:10.814480476 (5h0m24.401536717s) [upload=5160]: Got next segment. Inline: true Aug 7 21:27:10.814485826 (5h0m24.401542067s) [upload=5160]: Waiting for error group managing segments... Aug 7 21:27:10.814529677 (5h0m24.401585918s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:10.814535198 (5h0m24.401591438s) [upload=5160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:10.814538994 (5h0m24.401595236s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:40.714926118 (5h0m54.301982339s) [upload=5160]: Done uploading segments... Aug 7 21:27:40.714942067 (5h0m54.301998278s) [upload=5160]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:40.796846333 (5h0m54.383902554s) [upload=5169]: Starting upload Aug 7 21:27:40.796900224 (5h0m54.383956435s) [upload=5169]: Done starting upload Aug 7 21:27:40.796945107 (5h0m54.384001318s) [upload=5169]: Uploading segments... Aug 7 21:27:40.796979782 (5h0m54.384036003s) [upload=5169]: Got next segment. Inline: true Aug 7 21:27:40.796983369 (5h0m54.38403959s) [upload=5169]: Waiting for error group managing segments... Aug 7 21:27:40.797013695 (5h0m54.384069907s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:40.797017642 (5h0m54.384073863s) [upload=5169]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:40.797020007 (5h0m54.384076218s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:10.771421400 (5h1m24.358477651s) [upload=5169]: Done uploading segments... Aug 7 21:28:10.771479779 (5h1m24.35853602s) [upload=5169]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:23.151092147 (5h1m36.738148388s) [upload=5182]: Starting upload Aug 7 21:28:23.151190951 (5h1m36.738247193s) [upload=5182]: Done starting upload Aug 7 21:28:23.151342555 (5h1m36.738398796s) [upload=5182]: Uploading segments... Aug 7 21:28:23.151386697 (5h1m36.738442937s) [upload=5182]: Got next segment. Inline: true Aug 7 21:28:23.151392588 (5h1m36.738448829s) [upload=5182]: Waiting for error group managing segments... Aug 7 21:28:23.151436370 (5h1m36.738492611s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:23.151441740 (5h1m36.738497981s) [upload=5182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:23.151445377 (5h1m36.738501618s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:53.132183214 (5h2m6.719239455s) [upload=5182]: Done uploading segments... Aug 7 21:28:53.132205376 (5h2m6.719261617s) [upload=5182]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:29:03.059565675 (5h2m16.646621886s) [upload=5190]: Starting upload Aug 7 21:29:03.059620448 (5h2m16.646676659s) [upload=5190]: Done starting upload Aug 7 21:29:03.059662505 (5h2m16.646718727s) [upload=5190]: Uploading segments... Aug 7 21:29:03.059702511 (5h2m16.646758732s) [upload=5190]: Got next segment. Inline: true Aug 7 21:29:03.059707350 (5h2m16.646763561s) [upload=5190]: Waiting for error group managing segments... Aug 7 21:29:03.059743357 (5h2m16.646799578s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:03.059748677 (5h2m16.646804898s) [upload=5190]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:29:03.059752263 (5h2m16.646808474s) [upload=5190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:33.039700593 (5h2m46.626756834s) [upload=5190]: Done uploading segments... Aug 7 21:29:33.039722394 (5h2m46.626778635s) [upload=5190]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.001220606s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.000421117s FAIL (2025-08-07 21:29:31 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== lush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Aug 7 21:22:02.000538540 (4h55m15.587594761s) [upload=5082]: Starting upload Aug 7 21:22:02.000624420 (4h55m15.587680652s) [upload=5082]: Done starting upload Aug 7 21:22:02.000735858 (4h55m15.587792089s) [upload=5082]: Uploading segments... Aug 7 21:22:02.000881511 (4h55m15.587937761s) [upload=5082]: Got next segment. Inline: false Aug 7 21:22:02.000947453 (4h55m15.588003684s) [upload=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Aug 7 21:22:41.968187982 (4h55m55.555244194s) [upload=5089]: Starting upload Aug 7 21:22:41.968245641 (4h55m55.555301862s) [upload=5089]: Done starting upload Aug 7 21:22:41.968339846 (4h55m55.555396087s) [upload=5089]: Uploading segments... Aug 7 21:22:41.968596806 (4h55m55.555653047s) [upload=5089]: Got next segment. Inline: false Aug 7 21:22:41.968638122 (4h55m55.555694354s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Aug 7 21:23:20.066472018 (4h56m33.653528259s) [upload=5100]: Starting upload Aug 7 21:23:20.066538723 (4h56m33.653594965s) [upload=5100]: Done starting upload Aug 7 21:23:20.066650672 (4h56m33.653706913s) [upload=5100]: Uploading segments... Aug 7 21:23:20.066882696 (4h56m33.653938936s) [upload=5100]: Got next segment. Inline: false Aug 7 21:23:20.066929522 (4h56m33.653985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Aug 7 21:23:50.132503139 (4h57m3.719559361s) [upload=5111]: Starting upload Aug 7 21:23:50.132542101 (4h57m3.719598322s) [upload=5111]: Done starting upload Aug 7 21:23:50.132574512 (4h57m3.719630733s) [upload=5111]: Uploading segments... Aug 7 21:23:50.132772312 (4h57m3.719828533s) [upload=5111]: Got next segment. Inline: false Aug 7 21:23:50.132810694 (4h57m3.719866904s) [upload=5111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:50.132814390 (4h57m3.719870611s) [upload=5111]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:20.119434924 (4h57m33.706491135s) [upload=5111]: Next segment err: context deadline exceeded Aug 7 21:24:20.119485478 (4h57m33.706541699s) [upload=5111]: Done uploading segments... Aug 7 21:24:20.119497891 (4h57m33.706554102s) [upload=5111]: Upload finished. err: context deadline exceeded Aug 7 21:24:45.494243280 (4h57m59.081299501s) [upload=5120]: Starting upload Aug 7 21:24:45.494313381 (4h57m59.081369612s) [upload=5120]: Done starting upload Aug 7 21:24:45.494474943 (4h57m59.081531184s) [upload=5120]: Uploading segments... Aug 7 21:24:45.494627989 (4h57m59.08168422s) [upload=5120]: Got next segment. Inline: false Aug 7 21:24:45.494663625 (4h57m59.081719856s) [upload=5120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:45.494668073 (4h57m59.081724294s) [upload=5120]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:15.480538548 (4h58m29.067594769s) [upload=5120]: Next segment err: context deadline exceeded Aug 7 21:25:15.480665896 (4h58m29.067722117s) [upload=5120]: Done uploading segments... Aug 7 21:25:15.480717081 (4h58m29.067773302s) [upload=5120]: Upload finished. err: context deadline exceeded Aug 7 21:25:15.571225478 (4h58m29.158281699s) [upload=5129]: Starting upload Aug 7 21:25:15.571281894 (4h58m29.158338105s) [upload=5129]: Done starting upload Aug 7 21:25:15.571359468 (4h58m29.158415689s) [upload=5129]: Uploading segments... Aug 7 21:25:15.571564180 (4h58m29.158620401s) [upload=5129]: Got next segment. Inline: false Aug 7 21:25:15.571622810 (4h58m29.158679021s) [upload=5129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.571628479 (4h58m29.1586847s) [upload=5129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:45.547010083 (4h58m59.134066294s) [upload=5129]: Next segment err: context deadline exceeded Aug 7 21:25:45.547084462 (4h58m59.134140683s) [upload=5129]: Done uploading segments... Aug 7 21:25:45.547107044 (4h58m59.134163266s) [upload=5129]: Upload finished. err: context deadline exceeded Aug 7 21:25:45.641521062 (4h58m59.228577283s) [upload=5137]: Starting upload Aug 7 21:25:45.641578831 (4h58m59.228635041s) [upload=5137]: Done starting upload Aug 7 21:25:45.641631238 (4h58m59.228687459s) [upload=5137]: Uploading segments... Aug 7 21:25:45.641794333 (4h58m59.228850554s) [upload=5137]: Got next segment. Inline: false Aug 7 21:25:45.641818608 (4h58m59.228874829s) [upload=5137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.641822736 (4h58m59.228878947s) [upload=5137]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:15.630688562 (4h59m29.217744793s) [upload=5137]: Next segment err: context deadline exceeded Aug 7 21:26:15.630873647 (4h59m29.217929858s) [upload=5137]: Done uploading segments... Aug 7 21:26:15.630892703 (4h59m29.217948914s) [upload=5137]: Upload finished. err: context deadline exceeded Aug 7 21:26:29.805240268 (4h59m43.392296479s) [upload=5148]: Starting upload Aug 7 21:26:29.805289190 (4h59m43.392345411s) [upload=5148]: Done starting upload Aug 7 21:26:29.805393004 (4h59m43.392449225s) [upload=5148]: Uploading segments... Aug 7 21:26:29.805516605 (4h59m43.392572816s) [upload=5148]: Got next segment. Inline: false Aug 7 21:26:29.805549756 (4h59m43.392605967s) [upload=5148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.805553633 (4h59m43.392609854s) [upload=5148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:59.790298417 (5h0m13.377354628s) [upload=5148]: Next segment err: context deadline exceeded Aug 7 21:26:59.790450801 (5h0m13.377507012s) [upload=5148]: Done uploading segments... Aug 7 21:26:59.790479374 (5h0m13.377535595s) [upload=5148]: Upload finished. err: context deadline exceeded Aug 7 21:26:59.854535904 (5h0m13.441592145s) [upload=5156]: Starting upload Aug 7 21:26:59.854627225 (5h0m13.441683465s) [upload=5156]: Done starting upload Aug 7 21:26:59.854723153 (5h0m13.441779394s) [upload=5156]: Uploading segments... Aug 7 21:26:59.855007455 (5h0m13.442063705s) [upload=5156]: Got next segment. Inline: false Aug 7 21:26:59.855059541 (5h0m13.442115782s) [upload=5156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:59.855096871 (5h0m13.442153123s) [upload=5156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:27:29.841757946 (5h0m43.428814187s) [upload=5156]: Next segment err: context deadline exceeded Aug 7 21:27:29.841960274 (5h0m43.429016514s) [upload=5156]: Done uploading segments... Aug 7 21:27:29.842007351 (5h0m43.429063593s) [upload=5156]: Upload finished. err: context deadline exceeded Aug 7 21:28:06.370955849 (5h1m19.95801209s) [upload=5175]: Starting upload Aug 7 21:28:06.371030067 (5h1m19.958086308s) [upload=5175]: Done starting upload Aug 7 21:28:06.371151734 (5h1m19.958207975s) [upload=5175]: Uploading segments... Aug 7 21:28:06.371451254 (5h1m19.958507495s) [upload=5175]: Got next segment. Inline: false Aug 7 21:28:06.371506767 (5h1m19.958563008s) [upload=5175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:06.371512959 (5h1m19.958569209s) [upload=5175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:28:36.358597390 (5h1m49.945653632s) [upload=5175]: Next segment err: context deadline exceeded Aug 7 21:28:36.358673913 (5h1m49.945730154s) [upload=5175]: Done uploading segments... Aug 7 21:28:36.358704921 (5h1m49.945761162s) [upload=5175]: Upload finished. err: context deadline exceeded Aug 7 21:29:01.102431480 (5h2m14.689487702s) [upload=5189]: Starting upload Aug 7 21:29:01.102546615 (5h2m14.689602836s) [upload=5189]: Done starting upload Aug 7 21:29:01.102685454 (5h2m14.689741675s) [upload=5189]: Uploading segments... Aug 7 21:29:01.102866763 (5h2m14.689922984s) [upload=5189]: Got next segment. Inline: false Aug 7 21:29:01.102903291 (5h2m14.689959502s) [upload=5189]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:29:01.102907839 (5h2m14.68996406s) [upload=5189]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:29:31.087349640 (5h2m44.67440585s) [upload=5189]: Next segment err: context deadline exceeded Aug 7 21:29:31.087429919 (5h2m44.67448613s) [upload=5189]: Done uploading segments... Aug 7 21:29:31.087444957 (5h2m44.674501178s) [upload=5189]: Upload finished. err: context deadline exceeded Duration: 30.000436585s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 22 out of 60 is failed (threshold=3) [....................................XX..XXXXXXXXXXXXXXXXXXXX] duration: 30.001104251s FAIL (2025-08-07 21:29:31 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001123888s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.229262086s FAIL (2025-08-07 21:29:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.22929182s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 30.000479296s FAIL (2025-08-07 21:29:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000490536s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.001421545s FAIL (2025-08-07 21:29:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001435541s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 16 out of 60 is failed (threshold=3) [...........................................XX.XXXXXXXXXXXXXX] duration: 1m0.000755056s FAIL (2025-08-07 21:29:18 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000958908s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 812.748812ms FAIL (2025-08-07 21:29:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 812.771364ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 1m0.000690943s FAIL (2025-08-07 21:29:05 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e exceeded Aug 7 21:18:29.988006379 (4h51m43.5750626s) [upload=5023]: Starting upload Aug 7 21:18:29.988053607 (4h51m43.575109818s) [upload=5023]: Done starting upload Aug 7 21:18:29.988109782 (4h51m43.575166003s) [upload=5023]: Uploading segments... Aug 7 21:18:29.988142012 (4h51m43.575198224s) [upload=5023]: Got next segment. Inline: true Aug 7 21:18:29.988147753 (4h51m43.575203973s) [upload=5023]: Waiting for error group managing segments... Aug 7 21:18:29.988197475 (4h51m43.575253687s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:29.988203116 (4h51m43.575259327s) [upload=5023]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:29.988206943 (4h51m43.575263164s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:29.967158594 (4h52m43.554214836s) [upload=5023]: Done uploading segments... Aug 7 21:19:29.967184642 (4h52m43.554240884s) [upload=5023]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:30.042411605 (4h52m43.629467826s) [upload=5040]: Starting upload Aug 7 21:19:30.042491373 (4h52m43.629547594s) [upload=5040]: Done starting upload Aug 7 21:19:30.042544984 (4h52m43.629601195s) [upload=5040]: Uploading segments... Aug 7 21:19:30.042585870 (4h52m43.629642091s) [upload=5040]: Got next segment. Inline: true Aug 7 21:19:30.042590809 (4h52m43.62964703s) [upload=5040]: Waiting for error group managing segments... Aug 7 21:19:30.042637226 (4h52m43.629693436s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:30.042642325 (4h52m43.629698546s) [upload=5040]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:30.042647144 (4h52m43.629703365s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:30.020861737 (4h53m43.607917958s) [upload=5040]: Done uploading segments... Aug 7 21:20:30.020878959 (4h53m43.60793518s) [upload=5040]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:30.189934289 (4h54m43.776990511s) [upload=5071]: Starting upload Aug 7 21:21:30.189980476 (4h54m43.777036687s) [upload=5071]: Done starting upload Aug 7 21:21:30.190017424 (4h54m43.777073645s) [upload=5071]: Uploading segments... Aug 7 21:21:30.190043793 (4h54m43.777100005s) [upload=5071]: Got next segment. Inline: true Aug 7 21:21:30.190047100 (4h54m43.777103321s) [upload=5071]: Waiting for error group managing segments... Aug 7 21:21:30.190088458 (4h54m43.777144669s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:30.190092425 (4h54m43.777148646s) [upload=5071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:30.190094709 (4h54m43.77715093s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:30.179283539 (4h55m43.76633979s) [upload=5071]: Done uploading segments... Aug 7 21:22:30.179305229 (4h55m43.766361471s) [upload=5071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.041348848 (4h55m57.628405089s) [upload=5090]: Starting upload Aug 7 21:22:44.041428967 (4h55m57.628485208s) [upload=5090]: Done starting upload Aug 7 21:22:44.041486454 (4h55m57.628542695s) [upload=5090]: Uploading segments... Aug 7 21:22:44.041542930 (4h55m57.628599171s) [upload=5090]: Got next segment. Inline: true Aug 7 21:22:44.041549963 (4h55m57.628606204s) [upload=5090]: Waiting for error group managing segments... Aug 7 21:22:44.041596389 (4h55m57.628652631s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.041602170 (4h55m57.628658411s) [upload=5090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.041605667 (4h55m57.628661917s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.025768658 (4h56m57.612824879s) [upload=5090]: Done uploading segments... Aug 7 21:23:44.025784877 (4h56m57.612841089s) [upload=5090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:58.208173683 (4h57m11.795229924s) [upload=5112]: Starting upload Aug 7 21:23:58.208257669 (4h57m11.795313911s) [upload=5112]: Done starting upload Aug 7 21:23:58.208337199 (4h57m11.79539344s) [upload=5112]: Uploading segments... Aug 7 21:23:58.208378025 (4h57m11.795434265s) [upload=5112]: Got next segment. Inline: true Aug 7 21:23:58.208383675 (4h57m11.795439916s) [upload=5112]: Waiting for error group managing segments... Aug 7 21:23:58.208427437 (4h57m11.795483678s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:58.208433278 (4h57m11.795489519s) [upload=5112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:58.208438388 (4h57m11.795494628s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:58.195766004 (4h58m11.782822225s) [upload=5112]: Done uploading segments... Aug 7 21:24:58.195780731 (4h58m11.782836943s) [upload=5112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:04.854742735 (4h58m18.441798946s) [upload=5127]: Starting upload Aug 7 21:25:04.854807696 (4h58m18.441863907s) [upload=5127]: Done starting upload Aug 7 21:25:04.854870773 (4h58m18.441926984s) [upload=5127]: Uploading segments... Aug 7 21:25:04.854916199 (4h58m18.44197242s) [upload=5127]: Got next segment. Inline: true Aug 7 21:25:04.854936166 (4h58m18.441992387s) [upload=5127]: Waiting for error group managing segments... Aug 7 21:25:04.854972444 (4h58m18.442028655s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:04.854976711 (4h58m18.442032922s) [upload=5127]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:04.854979997 (4h58m18.442036218s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:04.841433540 (4h59m18.428489761s) [upload=5127]: Done uploading segments... Aug 7 21:26:04.841457825 (4h59m18.428514037s) [upload=5127]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:04.958090745 (4h59m18.545146957s) [upload=5139]: Starting upload Aug 7 21:26:04.958138415 (4h59m18.545194626s) [upload=5139]: Done starting upload Aug 7 21:26:04.958184982 (4h59m18.545241203s) [upload=5139]: Uploading segments... Aug 7 21:26:04.958220368 (4h59m18.545276589s) [upload=5139]: Got next segment. Inline: true Aug 7 21:26:04.958224295 (4h59m18.545280516s) [upload=5139]: Waiting for error group managing segments... Aug 7 21:26:04.958253820 (4h59m18.545310031s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:04.958257427 (4h59m18.545313638s) [upload=5139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:04.958259772 (4h59m18.545315993s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:04.939553376 (5h0m18.526609617s) [upload=5139]: Done uploading segments... Aug 7 21:27:04.939574906 (5h0m18.526631147s) [upload=5139]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:05.045401846 (5h0m18.632458067s) [upload=5157]: Starting upload Aug 7 21:27:05.045447422 (5h0m18.632503643s) [upload=5157]: Done starting upload Aug 7 21:27:05.045500470 (5h0m18.632556691s) [upload=5157]: Uploading segments... Aug 7 21:27:05.045530256 (5h0m18.632586467s) [upload=5157]: Got next segment. Inline: true Aug 7 21:27:05.045533973 (5h0m18.632590194s) [upload=5157]: Waiting for error group managing segments... Aug 7 21:27:05.045565873 (5h0m18.632622094s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:05.045571213 (5h0m18.632627434s) [upload=5157]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:05.045573487 (5h0m18.632629708s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:05.013376070 (5h1m18.600432281s) [upload=5157]: Done uploading segments... Aug 7 21:28:05.013391970 (5h1m18.600448181s) [upload=5157]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:05.107129855 (5h1m18.694186067s) [upload=5174]: Starting upload Aug 7 21:28:05.107184377 (5h1m18.694240598s) [upload=5174]: Done starting upload Aug 7 21:28:05.107240842 (5h1m18.694297063s) [upload=5174]: Uploading segments... Aug 7 21:28:05.107277301 (5h1m18.694333512s) [upload=5174]: Got next segment. Inline: true Aug 7 21:28:05.107280647 (5h1m18.694336868s) [upload=5174]: Waiting for error group managing segments... Aug 7 21:28:05.107312757 (5h1m18.694368969s) [upload=5174]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:05.107316373 (5h1m18.694372584s) [upload=5174]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:05.107318867 (5h1m18.694375088s) [upload=5174]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:29:05.085592777 (5h2m18.672648989s) [upload=5174]: Done uploading segments... Aug 7 21:29:05.085611523 (5h2m18.672667733s) [upload=5174]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.000702987s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 30.000579235s FAIL (2025-08-07 21:28:53 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== text deadline exceeded Aug 7 21:21:41.028727083 (4h54m54.615783293s) [upload=5077]: Starting upload Aug 7 21:21:41.028776044 (4h54m54.615832265s) [upload=5077]: Done starting upload Aug 7 21:21:41.028822871 (4h54m54.615879093s) [upload=5077]: Uploading segments... Aug 7 21:21:41.028849481 (4h54m54.615905693s) [upload=5077]: Got next segment. Inline: true Aug 7 21:21:41.028853158 (4h54m54.61590938s) [upload=5077]: Waiting for error group managing segments... Aug 7 21:21:41.028887172 (4h54m54.615943383s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:41.028891160 (4h54m54.615947381s) [upload=5077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:41.028893564 (4h54m54.615949785s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:11.020606617 (4h55m24.607662858s) [upload=5077]: Done uploading segments... Aug 7 21:22:11.020629420 (4h55m24.60768566s) [upload=5077]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.704346518 (4h55m58.291402739s) [upload=5092]: Starting upload Aug 7 21:22:44.704395569 (4h55m58.29145179s) [upload=5092]: Done starting upload Aug 7 21:22:44.704432468 (4h55m58.291488689s) [upload=5092]: Uploading segments... Aug 7 21:22:44.704457685 (4h55m58.291513906s) [upload=5092]: Got next segment. Inline: true Aug 7 21:22:44.704461182 (4h55m58.291517403s) [upload=5092]: Waiting for error group managing segments... Aug 7 21:22:44.704487161 (4h55m58.291543382s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.704490647 (4h55m58.291546868s) [upload=5092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.704493612 (4h55m58.291549833s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.694775680 (4h56m28.281831901s) [upload=5092]: Done uploading segments... Aug 7 21:23:14.694790848 (4h56m28.281847069s) [upload=5092]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.781991861 (4h56m28.369048072s) [upload=5099]: Starting upload Aug 7 21:23:14.782035392 (4h56m28.369091604s) [upload=5099]: Done starting upload Aug 7 21:23:14.782087559 (4h56m28.369143771s) [upload=5099]: Uploading segments... Aug 7 21:23:14.782124759 (4h56m28.369180991s) [upload=5099]: Got next segment. Inline: true Aug 7 21:23:14.782130029 (4h56m28.369186241s) [upload=5099]: Waiting for error group managing segments... Aug 7 21:23:14.782161368 (4h56m28.369217579s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.782164914 (4h56m28.369221126s) [upload=5099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.782167850 (4h56m28.369224061s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.771505754 (4h56m58.358561975s) [upload=5099]: Done uploading segments... Aug 7 21:23:44.771536371 (4h56m58.358592583s) [upload=5099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:44.852200021 (4h56m58.439256242s) [upload=5107]: Starting upload Aug 7 21:23:44.852251448 (4h56m58.439307669s) [upload=5107]: Done starting upload Aug 7 21:23:44.852291703 (4h56m58.439347924s) [upload=5107]: Uploading segments... Aug 7 21:23:44.852320757 (4h56m58.439376978s) [upload=5107]: Got next segment. Inline: true Aug 7 21:23:44.852324224 (4h56m58.439380445s) [upload=5107]: Waiting for error group managing segments... Aug 7 21:23:44.852355542 (4h56m58.439411753s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:44.852359279 (4h56m58.4394155s) [upload=5107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:44.852361614 (4h56m58.439417824s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.835050939 (4h57m28.42210715s) [upload=5107]: Done uploading segments... Aug 7 21:24:14.835115480 (4h57m28.422171701s) [upload=5107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:56.715637130 (4h58m10.302693351s) [upload=5124]: Starting upload Aug 7 21:24:56.715691522 (4h58m10.302747743s) [upload=5124]: Done starting upload Aug 7 21:24:56.715738660 (4h58m10.302794881s) [upload=5124]: Uploading segments... Aug 7 21:24:56.715821674 (4h58m10.302877895s) [upload=5124]: Got next segment. Inline: true Aug 7 21:24:56.715825752 (4h58m10.302881973s) [upload=5124]: Waiting for error group managing segments... Aug 7 21:24:56.715858083 (4h58m10.302914294s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:56.715861810 (4h58m10.302918031s) [upload=5124]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:56.715864014 (4h58m10.302920235s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:26.698879559 (4h58m40.285935771s) [upload=5124]: Done uploading segments... Aug 7 21:25:26.698896120 (4h58m40.285952331s) [upload=5124]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:27.243732093 (4h58m40.830788304s) [upload=5132]: Starting upload Aug 7 21:25:27.243787377 (4h58m40.830843598s) [upload=5132]: Done starting upload Aug 7 21:25:27.243846396 (4h58m40.830902607s) [upload=5132]: Uploading segments... Aug 7 21:25:27.243883505 (4h58m40.830939727s) [upload=5132]: Got next segment. Inline: true Aug 7 21:25:27.243887303 (4h58m40.830943524s) [upload=5132]: Waiting for error group managing segments... Aug 7 21:25:27.243942507 (4h58m40.830998728s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:27.243946624 (4h58m40.831002845s) [upload=5132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:27.243948898 (4h58m40.831005119s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.223887428 (4h59m10.810943669s) [upload=5132]: Done uploading segments... Aug 7 21:25:57.223909498 (4h59m10.81096574s) [upload=5132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:29.142150756 (4h59m42.729206977s) [upload=5147]: Starting upload Aug 7 21:26:29.142194398 (4h59m42.729250619s) [upload=5147]: Done starting upload Aug 7 21:26:29.142233742 (4h59m42.729289963s) [upload=5147]: Uploading segments... Aug 7 21:26:29.142260401 (4h59m42.729316612s) [upload=5147]: Got next segment. Inline: true Aug 7 21:26:29.142263868 (4h59m42.729320089s) [upload=5147]: Waiting for error group managing segments... Aug 7 21:26:29.142293002 (4h59m42.729349213s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.142297080 (4h59m42.729353301s) [upload=5147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:29.142299354 (4h59m42.729355575s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:59.127042625 (5h0m12.714098866s) [upload=5147]: Done uploading segments... Aug 7 21:26:59.127117424 (5h0m12.714173665s) [upload=5147]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:10.814290180 (5h0m24.401346421s) [upload=5160]: Starting upload Aug 7 21:27:10.814371522 (5h0m24.401427764s) [upload=5160]: Done starting upload Aug 7 21:27:10.814438597 (5h0m24.401494839s) [upload=5160]: Uploading segments... Aug 7 21:27:10.814480476 (5h0m24.401536717s) [upload=5160]: Got next segment. Inline: true Aug 7 21:27:10.814485826 (5h0m24.401542067s) [upload=5160]: Waiting for error group managing segments... Aug 7 21:27:10.814529677 (5h0m24.401585918s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:10.814535198 (5h0m24.401591438s) [upload=5160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:10.814538994 (5h0m24.401595236s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:40.714926118 (5h0m54.301982339s) [upload=5160]: Done uploading segments... Aug 7 21:27:40.714942067 (5h0m54.301998278s) [upload=5160]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:40.796846333 (5h0m54.383902554s) [upload=5169]: Starting upload Aug 7 21:27:40.796900224 (5h0m54.383956435s) [upload=5169]: Done starting upload Aug 7 21:27:40.796945107 (5h0m54.384001318s) [upload=5169]: Uploading segments... Aug 7 21:27:40.796979782 (5h0m54.384036003s) [upload=5169]: Got next segment. Inline: true Aug 7 21:27:40.796983369 (5h0m54.38403959s) [upload=5169]: Waiting for error group managing segments... Aug 7 21:27:40.797013695 (5h0m54.384069907s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:40.797017642 (5h0m54.384073863s) [upload=5169]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:40.797020007 (5h0m54.384076218s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:10.771421400 (5h1m24.358477651s) [upload=5169]: Done uploading segments... Aug 7 21:28:10.771479779 (5h1m24.35853602s) [upload=5169]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:28:23.151092147 (5h1m36.738148388s) [upload=5182]: Starting upload Aug 7 21:28:23.151190951 (5h1m36.738247193s) [upload=5182]: Done starting upload Aug 7 21:28:23.151342555 (5h1m36.738398796s) [upload=5182]: Uploading segments... Aug 7 21:28:23.151386697 (5h1m36.738442937s) [upload=5182]: Got next segment. Inline: true Aug 7 21:28:23.151392588 (5h1m36.738448829s) [upload=5182]: Waiting for error group managing segments... Aug 7 21:28:23.151436370 (5h1m36.738492611s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:23.151441740 (5h1m36.738497981s) [upload=5182]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:28:23.151445377 (5h1m36.738501618s) [upload=5182]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:53.132183214 (5h2m6.719239455s) [upload=5182]: Done uploading segments... Aug 7 21:28:53.132205376 (5h2m6.719261617s) [upload=5182]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000602428s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.000459264s FAIL (2025-08-07 21:28:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000484419s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 30.001219262s FAIL (2025-08-07 21:28:40 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001234851s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.00137679s FAIL (2025-08-07 21:28:36 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:54.290184501 (4h54m7.877240741s) [upload=5062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:21:24.274797304 (4h54m37.861853546s) [upload=5062]: Next segment err: context deadline exceeded Aug 7 21:21:24.274996005 (4h54m37.862052246s) [upload=5062]: Done uploading segments... Aug 7 21:21:24.275019569 (4h54m37.86207581s) [upload=5062]: Upload finished. err: context deadline exceeded Aug 7 21:21:31.936040213 (4h54m45.523096464s) [upload=5073]: Starting upload Aug 7 21:21:31.936133387 (4h54m45.523189638s) [upload=5073]: Done starting upload Aug 7 21:21:31.936256627 (4h54m45.523312868s) [upload=5073]: Uploading segments... Aug 7 21:21:31.936495583 (4h54m45.523551824s) [upload=5073]: Got next segment. Inline: false Aug 7 21:21:31.936532813 (4h54m45.523589053s) [upload=5073]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:31.936537992 (4h54m45.523594234s) [upload=5073]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:01.921915749 (4h55m15.50897196s) [upload=5073]: Next segment err: context deadline exceeded Aug 7 21:22:01.922047263 (4h55m15.509103475s) [upload=5073]: Done uploading segments... Aug 7 21:22:01.922079755 (4h55m15.509135976s) [upload=5073]: Upload finished. err: context deadline exceeded Aug 7 21:22:02.000538540 (4h55m15.587594761s) [upload=5082]: Starting upload Aug 7 21:22:02.000624420 (4h55m15.587680652s) [upload=5082]: Done starting upload Aug 7 21:22:02.000735858 (4h55m15.587792089s) [upload=5082]: Uploading segments... Aug 7 21:22:02.000881511 (4h55m15.587937761s) [upload=5082]: Got next segment. Inline: false Aug 7 21:22:02.000947453 (4h55m15.588003684s) [upload=5082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:02.000954897 (4h55m15.588011118s) [upload=5082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:22:31.979343661 (4h55m45.566399882s) [upload=5082]: Next segment err: context deadline exceeded Aug 7 21:22:31.979438208 (4h55m45.566494418s) [upload=5082]: Done uploading segments... Aug 7 21:22:31.979474655 (4h55m45.566530877s) [upload=5082]: Upload finished. err: context deadline exceeded Aug 7 21:22:41.968187982 (4h55m55.555244194s) [upload=5089]: Starting upload Aug 7 21:22:41.968245641 (4h55m55.555301862s) [upload=5089]: Done starting upload Aug 7 21:22:41.968339846 (4h55m55.555396087s) [upload=5089]: Uploading segments... Aug 7 21:22:41.968596806 (4h55m55.555653047s) [upload=5089]: Got next segment. Inline: false Aug 7 21:22:41.968638122 (4h55m55.555694354s) [upload=5089]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:41.968644665 (4h55m55.555700906s) [upload=5089]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:11.943981705 (4h56m25.531037956s) [upload=5089]: Next segment err: context deadline exceeded Aug 7 21:23:11.944159476 (4h56m25.531215718s) [upload=5089]: Done uploading segments... Aug 7 21:23:11.944182590 (4h56m25.53123883s) [upload=5089]: Upload finished. err: context deadline exceeded Aug 7 21:23:20.066472018 (4h56m33.653528259s) [upload=5100]: Starting upload Aug 7 21:23:20.066538723 (4h56m33.653594965s) [upload=5100]: Done starting upload Aug 7 21:23:20.066650672 (4h56m33.653706913s) [upload=5100]: Uploading segments... Aug 7 21:23:20.066882696 (4h56m33.653938936s) [upload=5100]: Got next segment. Inline: false Aug 7 21:23:20.066929522 (4h56m33.653985764s) [upload=5100]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:20.066934632 (4h56m33.653990873s) [upload=5100]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:23:50.045530632 (4h57m3.632586874s) [upload=5100]: Next segment err: context deadline exceeded Aug 7 21:23:50.045695360 (4h57m3.6327516s) [upload=5100]: Done uploading segments... Aug 7 21:23:50.045720867 (4h57m3.632777109s) [upload=5100]: Upload finished. err: context deadline exceeded Aug 7 21:23:50.132503139 (4h57m3.719559361s) [upload=5111]: Starting upload Aug 7 21:23:50.132542101 (4h57m3.719598322s) [upload=5111]: Done starting upload Aug 7 21:23:50.132574512 (4h57m3.719630733s) [upload=5111]: Uploading segments... Aug 7 21:23:50.132772312 (4h57m3.719828533s) [upload=5111]: Got next segment. Inline: false Aug 7 21:23:50.132810694 (4h57m3.719866904s) [upload=5111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:50.132814390 (4h57m3.719870611s) [upload=5111]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:24:20.119434924 (4h57m33.706491135s) [upload=5111]: Next segment err: context deadline exceeded Aug 7 21:24:20.119485478 (4h57m33.706541699s) [upload=5111]: Done uploading segments... Aug 7 21:24:20.119497891 (4h57m33.706554102s) [upload=5111]: Upload finished. err: context deadline exceeded Aug 7 21:24:45.494243280 (4h57m59.081299501s) [upload=5120]: Starting upload Aug 7 21:24:45.494313381 (4h57m59.081369612s) [upload=5120]: Done starting upload Aug 7 21:24:45.494474943 (4h57m59.081531184s) [upload=5120]: Uploading segments... Aug 7 21:24:45.494627989 (4h57m59.08168422s) [upload=5120]: Got next segment. Inline: false Aug 7 21:24:45.494663625 (4h57m59.081719856s) [upload=5120]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:45.494668073 (4h57m59.081724294s) [upload=5120]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:15.480538548 (4h58m29.067594769s) [upload=5120]: Next segment err: context deadline exceeded Aug 7 21:25:15.480665896 (4h58m29.067722117s) [upload=5120]: Done uploading segments... Aug 7 21:25:15.480717081 (4h58m29.067773302s) [upload=5120]: Upload finished. err: context deadline exceeded Aug 7 21:25:15.571225478 (4h58m29.158281699s) [upload=5129]: Starting upload Aug 7 21:25:15.571281894 (4h58m29.158338105s) [upload=5129]: Done starting upload Aug 7 21:25:15.571359468 (4h58m29.158415689s) [upload=5129]: Uploading segments... Aug 7 21:25:15.571564180 (4h58m29.158620401s) [upload=5129]: Got next segment. Inline: false Aug 7 21:25:15.571622810 (4h58m29.158679021s) [upload=5129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:15.571628479 (4h58m29.1586847s) [upload=5129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:25:45.547010083 (4h58m59.134066294s) [upload=5129]: Next segment err: context deadline exceeded Aug 7 21:25:45.547084462 (4h58m59.134140683s) [upload=5129]: Done uploading segments... Aug 7 21:25:45.547107044 (4h58m59.134163266s) [upload=5129]: Upload finished. err: context deadline exceeded Aug 7 21:25:45.641521062 (4h58m59.228577283s) [upload=5137]: Starting upload Aug 7 21:25:45.641578831 (4h58m59.228635041s) [upload=5137]: Done starting upload Aug 7 21:25:45.641631238 (4h58m59.228687459s) [upload=5137]: Uploading segments... Aug 7 21:25:45.641794333 (4h58m59.228850554s) [upload=5137]: Got next segment. Inline: false Aug 7 21:25:45.641818608 (4h58m59.228874829s) [upload=5137]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:45.641822736 (4h58m59.228878947s) [upload=5137]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:15.630688562 (4h59m29.217744793s) [upload=5137]: Next segment err: context deadline exceeded Aug 7 21:26:15.630873647 (4h59m29.217929858s) [upload=5137]: Done uploading segments... Aug 7 21:26:15.630892703 (4h59m29.217948914s) [upload=5137]: Upload finished. err: context deadline exceeded Aug 7 21:26:29.805240268 (4h59m43.392296479s) [upload=5148]: Starting upload Aug 7 21:26:29.805289190 (4h59m43.392345411s) [upload=5148]: Done starting upload Aug 7 21:26:29.805393004 (4h59m43.392449225s) [upload=5148]: Uploading segments... Aug 7 21:26:29.805516605 (4h59m43.392572816s) [upload=5148]: Got next segment. Inline: false Aug 7 21:26:29.805549756 (4h59m43.392605967s) [upload=5148]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.805553633 (4h59m43.392609854s) [upload=5148]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:26:59.790298417 (5h0m13.377354628s) [upload=5148]: Next segment err: context deadline exceeded Aug 7 21:26:59.790450801 (5h0m13.377507012s) [upload=5148]: Done uploading segments... Aug 7 21:26:59.790479374 (5h0m13.377535595s) [upload=5148]: Upload finished. err: context deadline exceeded Aug 7 21:26:59.854535904 (5h0m13.441592145s) [upload=5156]: Starting upload Aug 7 21:26:59.854627225 (5h0m13.441683465s) [upload=5156]: Done starting upload Aug 7 21:26:59.854723153 (5h0m13.441779394s) [upload=5156]: Uploading segments... Aug 7 21:26:59.855007455 (5h0m13.442063705s) [upload=5156]: Got next segment. Inline: false Aug 7 21:26:59.855059541 (5h0m13.442115782s) [upload=5156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:59.855096871 (5h0m13.442153123s) [upload=5156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:27:29.841757946 (5h0m43.428814187s) [upload=5156]: Next segment err: context deadline exceeded Aug 7 21:27:29.841960274 (5h0m43.429016514s) [upload=5156]: Done uploading segments... Aug 7 21:27:29.842007351 (5h0m43.429063593s) [upload=5156]: Upload finished. err: context deadline exceeded Aug 7 21:28:06.370955849 (5h1m19.95801209s) [upload=5175]: Starting upload Aug 7 21:28:06.371030067 (5h1m19.958086308s) [upload=5175]: Done starting upload Aug 7 21:28:06.371151734 (5h1m19.958207975s) [upload=5175]: Uploading segments... Aug 7 21:28:06.371451254 (5h1m19.958507495s) [upload=5175]: Got next segment. Inline: false Aug 7 21:28:06.371506767 (5h1m19.958563008s) [upload=5175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:28:06.371512959 (5h1m19.958569209s) [upload=5175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:28:36.358597390 (5h1m49.945653632s) [upload=5175]: Next segment err: context deadline exceeded Aug 7 21:28:36.358673913 (5h1m49.945730154s) [upload=5175]: Done uploading segments... Aug 7 21:28:36.358704921 (5h1m49.945761162s) [upload=5175]: Upload finished. err: context deadline exceeded Duration: 30.001399702s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.....................................XX..XXXXXXXXXXXXXXXXXXX] duration: 30.000815143s FAIL (2025-08-07 21:28:32 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000833457s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 30.000795776s FAIL (2025-08-07 21:28:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00083514s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [............................................XX.XXXXXXXXXXXXX] duration: 1m0.000771649s FAIL (2025-08-07 21:28:18 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000794061s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 30.001544367s FAIL (2025-08-07 21:28:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001564404s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 30.000831861s FAIL (2025-08-07 21:28:10 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e exceeded Aug 7 21:21:03.692382456 (4h54m17.279438677s) [upload=5064]: Starting upload Aug 7 21:21:03.692428231 (4h54m17.279484453s) [upload=5064]: Done starting upload Aug 7 21:21:03.692467736 (4h54m17.279523957s) [upload=5064]: Uploading segments... Aug 7 21:21:03.692495146 (4h54m17.279551367s) [upload=5064]: Got next segment. Inline: true Aug 7 21:21:03.692500066 (4h54m17.279556287s) [upload=5064]: Waiting for error group managing segments... Aug 7 21:21:03.692529661 (4h54m17.279585882s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:03.692533137 (4h54m17.279589349s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:03.692535481 (4h54m17.279591702s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:33.679184065 (4h54m47.266240287s) [upload=5064]: Done uploading segments... Aug 7 21:21:33.679200356 (4h54m47.266256568s) [upload=5064]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:41.028727083 (4h54m54.615783293s) [upload=5077]: Starting upload Aug 7 21:21:41.028776044 (4h54m54.615832265s) [upload=5077]: Done starting upload Aug 7 21:21:41.028822871 (4h54m54.615879093s) [upload=5077]: Uploading segments... Aug 7 21:21:41.028849481 (4h54m54.615905693s) [upload=5077]: Got next segment. Inline: true Aug 7 21:21:41.028853158 (4h54m54.61590938s) [upload=5077]: Waiting for error group managing segments... Aug 7 21:21:41.028887172 (4h54m54.615943383s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:41.028891160 (4h54m54.615947381s) [upload=5077]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:41.028893564 (4h54m54.615949785s) [upload=5077]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:11.020606617 (4h55m24.607662858s) [upload=5077]: Done uploading segments... Aug 7 21:22:11.020629420 (4h55m24.60768566s) [upload=5077]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.704346518 (4h55m58.291402739s) [upload=5092]: Starting upload Aug 7 21:22:44.704395569 (4h55m58.29145179s) [upload=5092]: Done starting upload Aug 7 21:22:44.704432468 (4h55m58.291488689s) [upload=5092]: Uploading segments... Aug 7 21:22:44.704457685 (4h55m58.291513906s) [upload=5092]: Got next segment. Inline: true Aug 7 21:22:44.704461182 (4h55m58.291517403s) [upload=5092]: Waiting for error group managing segments... Aug 7 21:22:44.704487161 (4h55m58.291543382s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.704490647 (4h55m58.291546868s) [upload=5092]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.704493612 (4h55m58.291549833s) [upload=5092]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:14.694775680 (4h56m28.281831901s) [upload=5092]: Done uploading segments... Aug 7 21:23:14.694790848 (4h56m28.281847069s) [upload=5092]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:14.781991861 (4h56m28.369048072s) [upload=5099]: Starting upload Aug 7 21:23:14.782035392 (4h56m28.369091604s) [upload=5099]: Done starting upload Aug 7 21:23:14.782087559 (4h56m28.369143771s) [upload=5099]: Uploading segments... Aug 7 21:23:14.782124759 (4h56m28.369180991s) [upload=5099]: Got next segment. Inline: true Aug 7 21:23:14.782130029 (4h56m28.369186241s) [upload=5099]: Waiting for error group managing segments... Aug 7 21:23:14.782161368 (4h56m28.369217579s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:14.782164914 (4h56m28.369221126s) [upload=5099]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:14.782167850 (4h56m28.369224061s) [upload=5099]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.771505754 (4h56m58.358561975s) [upload=5099]: Done uploading segments... Aug 7 21:23:44.771536371 (4h56m58.358592583s) [upload=5099]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:44.852200021 (4h56m58.439256242s) [upload=5107]: Starting upload Aug 7 21:23:44.852251448 (4h56m58.439307669s) [upload=5107]: Done starting upload Aug 7 21:23:44.852291703 (4h56m58.439347924s) [upload=5107]: Uploading segments... Aug 7 21:23:44.852320757 (4h56m58.439376978s) [upload=5107]: Got next segment. Inline: true Aug 7 21:23:44.852324224 (4h56m58.439380445s) [upload=5107]: Waiting for error group managing segments... Aug 7 21:23:44.852355542 (4h56m58.439411753s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:44.852359279 (4h56m58.4394155s) [upload=5107]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:44.852361614 (4h56m58.439417824s) [upload=5107]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:14.835050939 (4h57m28.42210715s) [upload=5107]: Done uploading segments... Aug 7 21:24:14.835115480 (4h57m28.422171701s) [upload=5107]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:24:56.715637130 (4h58m10.302693351s) [upload=5124]: Starting upload Aug 7 21:24:56.715691522 (4h58m10.302747743s) [upload=5124]: Done starting upload Aug 7 21:24:56.715738660 (4h58m10.302794881s) [upload=5124]: Uploading segments... Aug 7 21:24:56.715821674 (4h58m10.302877895s) [upload=5124]: Got next segment. Inline: true Aug 7 21:24:56.715825752 (4h58m10.302881973s) [upload=5124]: Waiting for error group managing segments... Aug 7 21:24:56.715858083 (4h58m10.302914294s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:24:56.715861810 (4h58m10.302918031s) [upload=5124]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:24:56.715864014 (4h58m10.302920235s) [upload=5124]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:26.698879559 (4h58m40.285935771s) [upload=5124]: Done uploading segments... Aug 7 21:25:26.698896120 (4h58m40.285952331s) [upload=5124]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:27.243732093 (4h58m40.830788304s) [upload=5132]: Starting upload Aug 7 21:25:27.243787377 (4h58m40.830843598s) [upload=5132]: Done starting upload Aug 7 21:25:27.243846396 (4h58m40.830902607s) [upload=5132]: Uploading segments... Aug 7 21:25:27.243883505 (4h58m40.830939727s) [upload=5132]: Got next segment. Inline: true Aug 7 21:25:27.243887303 (4h58m40.830943524s) [upload=5132]: Waiting for error group managing segments... Aug 7 21:25:27.243942507 (4h58m40.830998728s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:27.243946624 (4h58m40.831002845s) [upload=5132]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:27.243948898 (4h58m40.831005119s) [upload=5132]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:25:57.223887428 (4h59m10.810943669s) [upload=5132]: Done uploading segments... Aug 7 21:25:57.223909498 (4h59m10.81096574s) [upload=5132]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:29.142150756 (4h59m42.729206977s) [upload=5147]: Starting upload Aug 7 21:26:29.142194398 (4h59m42.729250619s) [upload=5147]: Done starting upload Aug 7 21:26:29.142233742 (4h59m42.729289963s) [upload=5147]: Uploading segments... Aug 7 21:26:29.142260401 (4h59m42.729316612s) [upload=5147]: Got next segment. Inline: true Aug 7 21:26:29.142263868 (4h59m42.729320089s) [upload=5147]: Waiting for error group managing segments... Aug 7 21:26:29.142293002 (4h59m42.729349213s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:29.142297080 (4h59m42.729353301s) [upload=5147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:29.142299354 (4h59m42.729355575s) [upload=5147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:59.127042625 (5h0m12.714098866s) [upload=5147]: Done uploading segments... Aug 7 21:26:59.127117424 (5h0m12.714173665s) [upload=5147]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:10.814290180 (5h0m24.401346421s) [upload=5160]: Starting upload Aug 7 21:27:10.814371522 (5h0m24.401427764s) [upload=5160]: Done starting upload Aug 7 21:27:10.814438597 (5h0m24.401494839s) [upload=5160]: Uploading segments... Aug 7 21:27:10.814480476 (5h0m24.401536717s) [upload=5160]: Got next segment. Inline: true Aug 7 21:27:10.814485826 (5h0m24.401542067s) [upload=5160]: Waiting for error group managing segments... Aug 7 21:27:10.814529677 (5h0m24.401585918s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:10.814535198 (5h0m24.401591438s) [upload=5160]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:10.814538994 (5h0m24.401595236s) [upload=5160]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:40.714926118 (5h0m54.301982339s) [upload=5160]: Done uploading segments... Aug 7 21:27:40.714942067 (5h0m54.301998278s) [upload=5160]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:40.796846333 (5h0m54.383902554s) [upload=5169]: Starting upload Aug 7 21:27:40.796900224 (5h0m54.383956435s) [upload=5169]: Done starting upload Aug 7 21:27:40.796945107 (5h0m54.384001318s) [upload=5169]: Uploading segments... Aug 7 21:27:40.796979782 (5h0m54.384036003s) [upload=5169]: Got next segment. Inline: true Aug 7 21:27:40.796983369 (5h0m54.38403959s) [upload=5169]: Waiting for error group managing segments... Aug 7 21:27:40.797013695 (5h0m54.384069907s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:40.797017642 (5h0m54.384073863s) [upload=5169]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:40.797020007 (5h0m54.384076218s) [upload=5169]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:10.771421400 (5h1m24.358477651s) [upload=5169]: Done uploading segments... Aug 7 21:28:10.771479779 (5h1m24.35853602s) [upload=5169]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.0008469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 30.000733477s FAIL (2025-08-07 21:28:10 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000753945s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.694185331s FAIL (2025-08-07 21:28:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.694210969s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 1m0.001232196s FAIL (2025-08-07 21:28:05 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 7 21:17:29.913551414 (4h50m43.500607635s) [upload=5005]: Starting upload Aug 7 21:17:29.913606246 (4h50m43.500662467s) [upload=5005]: Done starting upload Aug 7 21:17:29.913653174 (4h50m43.500709395s) [upload=5005]: Uploading segments... Aug 7 21:17:29.913680986 (4h50m43.500737207s) [upload=5005]: Got next segment. Inline: true Aug 7 21:17:29.913684613 (4h50m43.500740835s) [upload=5005]: Waiting for error group managing segments... Aug 7 21:17:29.913715249 (4h50m43.500771471s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:17:29.913719057 (4h50m43.500775278s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:17:29.913721331 (4h50m43.500777542s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:18:29.900370534 (4h51m43.487426745s) [upload=5005]: Done uploading segments... Aug 7 21:18:29.900390291 (4h51m43.487446542s) [upload=5005]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:18:29.988006379 (4h51m43.5750626s) [upload=5023]: Starting upload Aug 7 21:18:29.988053607 (4h51m43.575109818s) [upload=5023]: Done starting upload Aug 7 21:18:29.988109782 (4h51m43.575166003s) [upload=5023]: Uploading segments... Aug 7 21:18:29.988142012 (4h51m43.575198224s) [upload=5023]: Got next segment. Inline: true Aug 7 21:18:29.988147753 (4h51m43.575203973s) [upload=5023]: Waiting for error group managing segments... Aug 7 21:18:29.988197475 (4h51m43.575253687s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:18:29.988203116 (4h51m43.575259327s) [upload=5023]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:18:29.988206943 (4h51m43.575263164s) [upload=5023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:19:29.967158594 (4h52m43.554214836s) [upload=5023]: Done uploading segments... Aug 7 21:19:29.967184642 (4h52m43.554240884s) [upload=5023]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:19:30.042411605 (4h52m43.629467826s) [upload=5040]: Starting upload Aug 7 21:19:30.042491373 (4h52m43.629547594s) [upload=5040]: Done starting upload Aug 7 21:19:30.042544984 (4h52m43.629601195s) [upload=5040]: Uploading segments... Aug 7 21:19:30.042585870 (4h52m43.629642091s) [upload=5040]: Got next segment. Inline: true Aug 7 21:19:30.042590809 (4h52m43.62964703s) [upload=5040]: Waiting for error group managing segments... Aug 7 21:19:30.042637226 (4h52m43.629693436s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:19:30.042642325 (4h52m43.629698546s) [upload=5040]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:19:30.042647144 (4h52m43.629703365s) [upload=5040]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:20:30.020861737 (4h53m43.607917958s) [upload=5040]: Done uploading segments... Aug 7 21:20:30.020878959 (4h53m43.60793518s) [upload=5040]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:20:30.129088028 (4h53m43.71614424s) [upload=5055]: Starting upload Aug 7 21:20:30.129144182 (4h53m43.716200404s) [upload=5055]: Done starting upload Aug 7 21:20:30.129191110 (4h53m43.716247332s) [upload=5055]: Uploading segments... Aug 7 21:20:30.129220234 (4h53m43.716276455s) [upload=5055]: Got next segment. Inline: true Aug 7 21:20:30.129223841 (4h53m43.716280062s) [upload=5055]: Waiting for error group managing segments... Aug 7 21:20:30.129266070 (4h53m43.716322282s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:20:30.129269998 (4h53m43.716326219s) [upload=5055]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:20:30.129272523 (4h53m43.716328734s) [upload=5055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:21:30.117153252 (4h54m43.704209473s) [upload=5055]: Done uploading segments... Aug 7 21:21:30.117172058 (4h54m43.704228279s) [upload=5055]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:21:30.189934289 (4h54m43.776990511s) [upload=5071]: Starting upload Aug 7 21:21:30.189980476 (4h54m43.777036687s) [upload=5071]: Done starting upload Aug 7 21:21:30.190017424 (4h54m43.777073645s) [upload=5071]: Uploading segments... Aug 7 21:21:30.190043793 (4h54m43.777100005s) [upload=5071]: Got next segment. Inline: true Aug 7 21:21:30.190047100 (4h54m43.777103321s) [upload=5071]: Waiting for error group managing segments... Aug 7 21:21:30.190088458 (4h54m43.777144669s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:21:30.190092425 (4h54m43.777148646s) [upload=5071]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:21:30.190094709 (4h54m43.77715093s) [upload=5071]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:22:30.179283539 (4h55m43.76633979s) [upload=5071]: Done uploading segments... Aug 7 21:22:30.179305229 (4h55m43.766361471s) [upload=5071]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:22:44.041348848 (4h55m57.628405089s) [upload=5090]: Starting upload Aug 7 21:22:44.041428967 (4h55m57.628485208s) [upload=5090]: Done starting upload Aug 7 21:22:44.041486454 (4h55m57.628542695s) [upload=5090]: Uploading segments... Aug 7 21:22:44.041542930 (4h55m57.628599171s) [upload=5090]: Got next segment. Inline: true Aug 7 21:22:44.041549963 (4h55m57.628606204s) [upload=5090]: Waiting for error group managing segments... Aug 7 21:22:44.041596389 (4h55m57.628652631s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:22:44.041602170 (4h55m57.628658411s) [upload=5090]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:22:44.041605667 (4h55m57.628661917s) [upload=5090]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:23:44.025768658 (4h56m57.612824879s) [upload=5090]: Done uploading segments... Aug 7 21:23:44.025784877 (4h56m57.612841089s) [upload=5090]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:23:58.208173683 (4h57m11.795229924s) [upload=5112]: Starting upload Aug 7 21:23:58.208257669 (4h57m11.795313911s) [upload=5112]: Done starting upload Aug 7 21:23:58.208337199 (4h57m11.79539344s) [upload=5112]: Uploading segments... Aug 7 21:23:58.208378025 (4h57m11.795434265s) [upload=5112]: Got next segment. Inline: true Aug 7 21:23:58.208383675 (4h57m11.795439916s) [upload=5112]: Waiting for error group managing segments... Aug 7 21:23:58.208427437 (4h57m11.795483678s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:23:58.208433278 (4h57m11.795489519s) [upload=5112]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:23:58.208438388 (4h57m11.795494628s) [upload=5112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:24:58.195766004 (4h58m11.782822225s) [upload=5112]: Done uploading segments... Aug 7 21:24:58.195780731 (4h58m11.782836943s) [upload=5112]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:25:04.854742735 (4h58m18.441798946s) [upload=5127]: Starting upload Aug 7 21:25:04.854807696 (4h58m18.441863907s) [upload=5127]: Done starting upload Aug 7 21:25:04.854870773 (4h58m18.441926984s) [upload=5127]: Uploading segments... Aug 7 21:25:04.854916199 (4h58m18.44197242s) [upload=5127]: Got next segment. Inline: true Aug 7 21:25:04.854936166 (4h58m18.441992387s) [upload=5127]: Waiting for error group managing segments... Aug 7 21:25:04.854972444 (4h58m18.442028655s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:25:04.854976711 (4h58m18.442032922s) [upload=5127]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:25:04.854979997 (4h58m18.442036218s) [upload=5127]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:26:04.841433540 (4h59m18.428489761s) [upload=5127]: Done uploading segments... Aug 7 21:26:04.841457825 (4h59m18.428514037s) [upload=5127]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:26:04.958090745 (4h59m18.545146957s) [upload=5139]: Starting upload Aug 7 21:26:04.958138415 (4h59m18.545194626s) [upload=5139]: Done starting upload Aug 7 21:26:04.958184982 (4h59m18.545241203s) [upload=5139]: Uploading segments... Aug 7 21:26:04.958220368 (4h59m18.545276589s) [upload=5139]: Got next segment. Inline: true Aug 7 21:26:04.958224295 (4h59m18.545280516s) [upload=5139]: Waiting for error group managing segments... Aug 7 21:26:04.958253820 (4h59m18.545310031s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:26:04.958257427 (4h59m18.545313638s) [upload=5139]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:26:04.958259772 (4h59m18.545315993s) [upload=5139]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:27:04.939553376 (5h0m18.526609617s) [upload=5139]: Done uploading segments... Aug 7 21:27:04.939574906 (5h0m18.526631147s) [upload=5139]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:27:05.045401846 (5h0m18.632458067s) [upload=5157]: Starting upload Aug 7 21:27:05.045447422 (5h0m18.632503643s) [upload=5157]: Done starting upload Aug 7 21:27:05.045500470 (5h0m18.632556691s) [upload=5157]: Uploading segments... Aug 7 21:27:05.045530256 (5h0m18.632586467s) [upload=5157]: Got next segment. Inline: true Aug 7 21:27:05.045533973 (5h0m18.632590194s) [upload=5157]: Waiting for error group managing segments... Aug 7 21:27:05.045565873 (5h0m18.632622094s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:27:05.045571213 (5h0m18.632627434s) [upload=5157]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:27:05.045573487 (5h0m18.632629708s) [upload=5157]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:28:05.013376070 (5h1m18.600432281s) [upload=5157]: Done uploading segments... Aug 7 21:28:05.013391970 (5h1m18.600448181s) [upload=5157]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.001243808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 20 out of 60 is failed (threshold=3) [......................................XX..XXXXXXXXXXXXXXXXXX] duration: 30.001084048s FAIL (2025-08-07 21:27:59 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00110605s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)