Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [.................................XXX.X.XXX.X..XX.X.X........] duration: 431.95377ms FAIL (2025-08-08 00:10:20 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 431.981192ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [......................................XXXXXXX...X...........] duration: 2.086236857s FAIL (2025-08-08 00:10:20 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.086257925s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 604.968498ms FAIL (2025-08-08 00:10:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 605.024103ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X...........X............] duration: 804.398265ms FAIL (2025-08-08 00:10:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 804.418392ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [........................................XX.X.......X........] duration: 643.425086ms FAIL (2025-08-08 00:10:15 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 643.449202ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................X.X..XXX.....X........] duration: 11.24287704s FAIL (2025-08-08 00:10:05 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 11.242899904s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X...........X...........] duration: 4.234297693s FAIL (2025-08-08 00:10:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.234320224s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [..................................XXX.X.XXX.X..XX.X.X.......] duration: 10.33750122s FAIL (2025-08-08 00:10:04 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 10.337527908s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................XXXXX.....X.X..........] duration: 465.750101ms FAIL (2025-08-08 00:09:52 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 465.784766ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...............................XXXXXXXXX...X................] duration: 205.5271ms FAIL (2025-08-08 00:09:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 205.546968ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 937.614986ms FAIL (2025-08-08 00:09:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 937.646674ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XX.X.......X.......] duration: 159.298576ms FAIL (2025-08-08 00:09:36 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 159.329334ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......................................XXXXX.....X.X.........] duration: 469.11546ms FAIL (2025-08-08 00:09:35 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 469.145065ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [.......................................XXXXXXX...X..........] duration: 1.480846266s FAIL (2025-08-08 00:09:33 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.480863699s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.X.........................................XX..X.X.........] duration: 942.145572ms FAIL (2025-08-08 00:09:28 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 942.176188ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 248.863599ms FAIL (2025-08-08 00:09:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 248.888195ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [................................XXXXXXXXX...X...............] duration: 189.899262ms FAIL (2025-08-08 00:09:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 189.936893ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.174986996s FAIL (2025-08-08 00:09:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.175012142s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X......................................XXXXX.....X.X........] duration: 377.790406ms FAIL (2025-08-08 00:09:18 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 377.812195ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 211.266303ms FAIL (2025-08-08 00:09:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 211.284969ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XX.X.......X......] duration: 416.402576ms FAIL (2025-08-08 00:09:10 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 416.467879ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [.................................XXXXXXXXX...X..............] duration: 283.532444ms FAIL (2025-08-08 00:09:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 283.56246ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X...........X..........] duration: 467.247917ms FAIL (2025-08-08 00:08:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 467.268786ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [........................................XXXXXXX...X.........] duration: 1.504983033s FAIL (2025-08-08 00:08:59 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.505001788s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................................X.X..XXX.....X.......] duration: 2.052674867s FAIL (2025-08-08 00:08:56 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.052700135s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...................................XXX.X.XXX.X..XX.X.X......] duration: 414.99654ms FAIL (2025-08-08 00:08:50 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 415.020735ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 188.437153ms FAIL (2025-08-08 00:08:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 188.460286ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X...........X.........] duration: 508.556078ms FAIL (2025-08-08 00:08:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 508.574783ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 756.760727ms FAIL (2025-08-08 00:08:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 756.788246ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 359.360107ms FAIL (2025-08-08 00:08:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 359.684493ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.X.........................................XX..X.X........] duration: 1.417292461s FAIL (2025-08-08 00:08:28 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 1.417321995s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [.........................................XXXXXXX...X........] duration: 1.407181297s FAIL (2025-08-08 00:08:19 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.407201585s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X......................................XXXXX.....X.X.......] duration: 1.220311992s FAIL (2025-08-08 00:08:06 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 1.220351784s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X...........X........] duration: 521.767758ms FAIL (2025-08-08 00:08:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 521.784169ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [..................................XXXXXXXXX...X.............] duration: 241.488169ms FAIL (2025-08-08 00:08:01 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 241.509861ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 630.755469ms FAIL (2025-08-08 00:08:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 630.780997ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XX.X.......X.....] duration: 176.13441ms FAIL (2025-08-08 00:07:57 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 176.159627ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 356.267155ms FAIL (2025-08-08 00:07:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 356.329931ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 154.294617ms FAIL (2025-08-08 00:07:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== estItem_SegmentMakeInline Aug 7 23:58:04.083340139 (7h31m17.670396381s) [upload=7709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:04.643858201 (7h31m18.230914422s) [upload=7709]: Done uploading segments... Aug 7 23:58:04.643875905 (7h31m18.230932126s) [upload=7709]: Upload finished. err: <nil> Aug 7 23:58:49.361770760 (7h32m2.948826981s) [upload=7722]: Starting upload Aug 7 23:58:49.361833366 (7h32m2.948889578s) [upload=7722]: Done starting upload Aug 7 23:58:49.361897717 (7h32m2.948953938s) [upload=7722]: Uploading segments... Aug 7 23:58:49.361929706 (7h32m2.948985927s) [upload=7722]: Got next segment. Inline: true Aug 7 23:58:49.361933683 (7h32m2.948989905s) [upload=7722]: Waiting for error group managing segments... Aug 7 23:58:49.361964291 (7h32m2.949020502s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:49.361967868 (7h32m2.949024079s) [upload=7722]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:49.361970062 (7h32m2.949026283s) [upload=7722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:49.419818133 (7h32m3.006874354s) [upload=7722]: Done uploading segments... Aug 7 23:58:49.419835917 (7h32m3.006892168s) [upload=7722]: Upload finished. err: <nil> Aug 7 23:59:16.473675329 (7h32m30.06073158s) [upload=7728]: Starting upload Aug 7 23:59:16.473772120 (7h32m30.060828361s) [upload=7728]: Done starting upload Aug 7 23:59:16.473856918 (7h32m30.060913159s) [upload=7728]: Uploading segments... Aug 7 23:59:16.473912993 (7h32m30.060969245s) [upload=7728]: Got next segment. Inline: true Aug 7 23:59:16.473921989 (7h32m30.06097824s) [upload=7728]: Waiting for error group managing segments... Aug 7 23:59:16.473988133 (7h32m30.061044374s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:16.473996829 (7h32m30.06105307s) [upload=7728]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:16.474003041 (7h32m30.061059292s) [upload=7728]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:16.554814633 (7h32m30.141870874s) [upload=7728]: Done uploading segments... Aug 7 23:59:16.554834199 (7h32m30.14189044s) [upload=7728]: Upload finished. err: <nil> Aug 8 00:00:24.825289162 (7h33m38.412345404s) [upload=7745]: Starting upload Aug 8 00:00:24.825385672 (7h33m38.412441923s) [upload=7745]: Done starting upload Aug 8 00:00:24.825495698 (7h33m38.412551949s) [upload=7745]: Uploading segments... Aug 8 00:00:24.825554116 (7h33m38.412610357s) [upload=7745]: Got next segment. Inline: true Aug 8 00:00:24.825562823 (7h33m38.412619074s) [upload=7745]: Waiting for error group managing segments... Aug 8 00:00:24.825631080 (7h33m38.412687332s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:24.825639566 (7h33m38.412695818s) [upload=7745]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:24.825645346 (7h33m38.412701638s) [upload=7745]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:24.942146694 (7h33m38.529202944s) [upload=7745]: Done uploading segments... Aug 8 00:00:24.942177611 (7h33m38.529233852s) [upload=7745]: Upload finished. err: <nil> Aug 8 00:01:17.985772683 (7h34m31.572828894s) [upload=7757]: Starting upload Aug 8 00:01:17.985821364 (7h34m31.572877585s) [upload=7757]: Done starting upload Aug 8 00:01:17.985881927 (7h34m31.572938138s) [upload=7757]: Uploading segments... Aug 8 00:01:17.985924556 (7h34m31.572980777s) [upload=7757]: Got next segment. Inline: true Aug 8 00:01:17.985945325 (7h34m31.573001536s) [upload=7757]: Waiting for error group managing segments... Aug 8 00:01:17.986001520 (7h34m31.573057741s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:17.986009444 (7h34m31.573065665s) [upload=7757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:17.986014143 (7h34m31.573070354s) [upload=7757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:18.060473510 (7h34m31.647529721s) [upload=7757]: Done uploading segments... Aug 8 00:01:18.060487786 (7h34m31.647543997s) [upload=7757]: Upload finished. err: <nil> Aug 8 00:01:33.275502426 (7h34m46.862558657s) [upload=7763]: Starting upload Aug 8 00:01:33.275558660 (7h34m46.862614881s) [upload=7763]: Done starting upload Aug 8 00:01:33.275623972 (7h34m46.862680203s) [upload=7763]: Uploading segments... Aug 8 00:01:33.275669688 (7h34m46.862725919s) [upload=7763]: Got next segment. Inline: true Aug 8 00:01:33.275673955 (7h34m46.862730186s) [upload=7763]: Waiting for error group managing segments... Aug 8 00:01:33.275713349 (7h34m46.86276957s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:33.275718068 (7h34m46.862774299s) [upload=7763]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:33.275720933 (7h34m46.862777165s) [upload=7763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:33.353696182 (7h34m46.940752424s) [upload=7763]: Done uploading segments... Aug 8 00:01:33.353716349 (7h34m46.94077259s) [upload=7763]: Upload finished. err: <nil> Aug 8 00:02:46.099697979 (7h35m59.68675428s) [upload=7783]: Starting upload Aug 8 00:02:46.099811050 (7h35m59.686867301s) [upload=7783]: Done starting upload Aug 8 00:02:46.099975548 (7h35m59.687031798s) [upload=7783]: Uploading segments... Aug 8 00:02:46.100038314 (7h35m59.687094585s) [upload=7783]: Got next segment. Inline: true Aug 8 00:02:46.100046931 (7h35m59.687103171s) [upload=7783]: Waiting for error group managing segments... Aug 8 00:02:46.100113385 (7h35m59.687169626s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:46.100122022 (7h35m59.687178262s) [upload=7783]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:46.100127581 (7h35m59.687183823s) [upload=7783]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:46.190100964 (7h35m59.777157204s) [upload=7783]: Done uploading segments... Aug 8 00:02:46.190119759 (7h35m59.777175999s) [upload=7783]: Upload finished. err: <nil> Aug 8 00:03:44.783527300 (7h36m58.370583521s) [upload=7796]: Starting upload Aug 8 00:03:44.783570200 (7h36m58.370626421s) [upload=7796]: Done starting upload Aug 8 00:03:44.783625023 (7h36m58.370681244s) [upload=7796]: Uploading segments... Aug 8 00:03:44.783650100 (7h36m58.370706311s) [upload=7796]: Got next segment. Inline: true Aug 8 00:03:44.783653726 (7h36m58.370709937s) [upload=7796]: Waiting for error group managing segments... Aug 8 00:03:44.783686457 (7h36m58.370742669s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:44.783690295 (7h36m58.370746505s) [upload=7796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:03:44.783692629 (7h36m58.37074885s) [upload=7796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:44.877401244 (7h36m58.464457465s) [upload=7796]: Done uploading segments... Aug 8 00:03:44.877414830 (7h36m58.464471051s) [upload=7796]: Upload finished. err: <nil> Aug 8 00:04:42.185900688 (7h37m55.772956929s) [upload=7815]: Starting upload Aug 8 00:04:42.185962914 (7h37m55.773019155s) [upload=7815]: Done starting upload Aug 8 00:04:42.186028456 (7h37m55.773084687s) [upload=7815]: Uploading segments... Aug 8 00:04:42.186077648 (7h37m55.773133889s) [upload=7815]: Got next segment. Inline: true Aug 8 00:04:42.186082998 (7h37m55.773139239s) [upload=7815]: Waiting for error group managing segments... Aug 8 00:04:42.186123654 (7h37m55.773179895s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:04:42.186128693 (7h37m55.773184935s) [upload=7815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:04:42.186131920 (7h37m55.77318816s) [upload=7815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:42.243666055 (7h37m55.830722356s) [upload=7815]: Done uploading segments... Aug 8 00:04:42.243690851 (7h37m55.830747102s) [upload=7815]: Upload finished. err: <nil> Aug 8 00:05:44.520710484 (7h38m58.107766705s) [upload=7831]: Starting upload Aug 8 00:05:44.520757893 (7h38m58.107814114s) [upload=7831]: Done starting upload Aug 8 00:05:44.520812865 (7h38m58.107869086s) [upload=7831]: Uploading segments... Aug 8 00:05:44.520846297 (7h38m58.107902509s) [upload=7831]: Got next segment. Inline: true Aug 8 00:05:44.520850204 (7h38m58.107906416s) [upload=7831]: Waiting for error group managing segments... Aug 8 00:05:44.520882675 (7h38m58.107938885s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:05:44.520886792 (7h38m58.107943013s) [upload=7831]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:05:44.520888996 (7h38m58.107945217s) [upload=7831]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:05:44.600687018 (7h38m58.187743239s) [upload=7831]: Done uploading segments... Aug 8 00:05:44.600702327 (7h38m58.187758548s) [upload=7831]: Upload finished. err: <nil> Aug 8 00:06:42.046759354 (7h39m55.633815575s) [upload=7848]: Starting upload Aug 8 00:06:42.046813624 (7h39m55.633869845s) [upload=7848]: Done starting upload Aug 8 00:06:42.046864920 (7h39m55.633921142s) [upload=7848]: Uploading segments... Aug 8 00:06:42.046909714 (7h39m55.633965945s) [upload=7848]: Got next segment. Inline: true Aug 8 00:06:42.046916086 (7h39m55.633972297s) [upload=7848]: Waiting for error group managing segments... Aug 8 00:06:42.046952323 (7h39m55.634008544s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:06:42.046956261 (7h39m55.634012482s) [upload=7848]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:06:42.046958956 (7h39m55.634015167s) [upload=7848]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:06:42.583893612 (7h39m56.170949852s) [upload=7848]: Done uploading segments... Aug 8 00:06:42.583933345 (7h39m56.170989576s) [upload=7848]: Upload finished. err: <nil> Duration:* 154.319934ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................................X.X..XXX.....X......] duration: 2.274121041s FAIL (2025-08-08 00:07:51 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.274150416s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [....................................XXX.X.XXX.X..XX.X.X.....] duration: 179.016028ms FAIL (2025-08-08 00:07:51 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 179.0327ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [X.X.X.........................................XX..X.X.......] duration: 932.458188ms FAIL (2025-08-08 00:07:46 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 932.488827ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [..........................................XXXXXXX...X.......] duration: 1.845068786s FAIL (2025-08-08 00:07:43 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.845088943s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 219.492876ms FAIL (2025-08-08 00:07:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 219.513204ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [XX.X.X.........................................XX..X.X......] duration: 1.225080441s FAIL (2025-08-08 00:07:15 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 943258937s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356509018 (7h35m11.943565239s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356527242 (7h35m11.943583453s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356781297 (7h35m11.943837508s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.356821863 (7h35m11.943878074s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357023649 (7h35m11.94407987s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357112525 (7h35m11.944168766s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357258177 (7h35m11.944314399s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357393539 (7h35m11.94444975s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357510978 (7h35m11.94456719s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357703989 (7h35m11.9447602s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357762478 (7h35m11.944818699s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.357995052 (7h35m11.945051263s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.358300603 (7h35m11.945356813s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.359247781 (7h35m11.946303992s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Uploading piece... Aug 8 00:01:58.440154511 (7h35m12.027210732s) [upload=7765]: Next returned nil segment Aug 8 00:01:58.440166041 (7h35m12.027222262s) [upload=7765]: Waiting for error group managing segments... Aug 8 00:01:58.461517814 (7h35m12.048574025s) [noise=true , address=104.36.82.19:4408 , piece_id=EHMQVB2AA57N2XIXGLFFKZHXKWZCHJV22NZNVYUD4RBUWWKUEMUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.461971391 (7h35m12.049027602s) [noise=true , address=45.91.116.4:4957 , piece_id=E2ZDTALTZSVEBXVWLGVD4KQPF6JIGXQ3DE33KAY7WCM7VNBAKJTA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464331726 (7h35m12.051387938s) [noise=true , address=104.36.82.38:4696 , piece_id=IGOIPFZV4IVK4FLLWMLQ3BLNZB26SIH4FDV4EMQB5MV2SUULIG4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464484571 (7h35m12.051540782s) [noise=true , address=45.91.116.13:4162 , piece_id=Z3KVGXPRDJ24VH3QCSCQQICNQTO4US4LAKYJLB2H3V2UKZDT6NKA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.464990136 (7h35m12.052046346s) [noise=true , address=104.36.82.62:4444 , piece_id=44UG7SBWAP3HNGXDH4UIXQNCYAGHC27NOHYGZTKH5TZFM5KNFDMQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.472019324 (7h35m12.059075546s) [noise=true , address=104.36.82.23:4463 , piece_id=EGQQOTR4PDBAIWQKYKLHUMG2GOOGXLMNEVZGN6OWRP6H5HG7SQGA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.476399520 (7h35m12.063455741s) [noise=true , address=45.91.116.9:4270 , piece_id=NOHOWXDJMNDFQ6OQ2J4LAXMIRMH6UZ4XQ45A553CUAJUYBOJCCUA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.488197731 (7h35m12.075253942s) [noise=true , address=104.36.82.9:4016 , piece_id=DZ2MCGFXRBWPN65T7VUCTJXATVCUVIF3INP6VWUV7H3UAQY3ILIA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.514321661 (7h35m12.101377882s) [noise=true , address=104.36.82.44:4883 , piece_id=EIVXE7DF4QR2SB2I4EAGODE3CUPSHNTSGQL4DL45T2F5KXFQ5ZXQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.543902222 (7h35m12.130958433s) [noise=true , address=104.36.82.61:4376 , piece_id=H2ARUZIF6PNPKZUJJRCTUSR3JO2YGEYAWHGGYUEOLDIFF3BAMA4A, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.917824890 (7h35m12.504881101s) [noise=true , address=104.36.82.47:4539 , piece_id=TNAW5MAEDTPXQKWAN5K3C7QWRDRG3QPUH623XWAPLSJ62JEGCCOA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:58.946029042 (7h35m12.533085253s) [noise=true , address=64.49.13.170:4724 , piece_id=46Q5IL4BYBK4CHP2WGYMNDKO33XZYGRSJ6WGRBXBOCFJPEHZSAYQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.080932867 (7h35m12.667989078s) [noise=true , address=104.36.82.37:4687 , piece_id=K7X3UFUV45DWLRF4ITY3LSAFQJBWAOGW3LF57I3YM7WPMENKO6UA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126038986 (7h35m12.713095207s) [noise=true , address=104.36.82.52:4087 , piece_id=L7IP6SEFH5D32K6VY4UF6QJTUZ5MMARTOVLNNGLAQCJFWMHXQJ4Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: <nil> Aug 8 00:01:59.126062560 (7h35m12.713118781s) [seg_pos={0 0}, upload=7765]: Segment reached optimal threshold of 14 pieces. Aug 8 00:01:59.126317085 (7h35m12.713373306s) [noise=true , address=104.36.82.60:4372 , piece_id=5JKKLQXSZT5EIVRWXL3CLWS7IDM6DRHJ5BPG2KQ5RF3P3WJBSYYA, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kJnsSPzFa7JwUMHuvosMmFrGvQ8tCsQbjwWdxbqjHNHgTfaL3): protocol: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10058->104.36.82.60:4372: use of closed network connection Aug 8 00:01:59.126359405 (7h35m12.713415626s) [noise=true , address=104.36.82.16:4324 , piece_id=CLTMB27LZSKFMS3UE5M5NVS4PNCAEAK4WTRIBGGK4X7XE7SSUKJQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12tmAP2k9LHn7tGVu5Ky9pLpwQ4aXettco3TRZ1fbmEC5SESkga): protocol: read tcp 5.161.89.252:38612->104.36.82.16:4324: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126382557 (7h35m12.713438779s) [noise=true , address=104.36.82.60:4344 , piece_id=7ORI6EQ7BHIW36TY2XIDTKRTKOY5637DCM2WUQK6RXSZ5QPDR3WQ, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:59454->104.36.82.60:4344: use of closed network connection Aug 8 00:01:59.126426129 (7h35m12.713482339s) [noise=true , address=104.36.82.45:4907 , piece_id=VJZYM54ZGK6MWYEKKFQ6BG45U5BPKL3GZSYCH3J6E3Y2UD5ZCI3Q, seg_pos={0 0}, upload=7765]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: read tcp 5.161.89.252:33482->104.36.82.45:4907: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:01:59.126459972 (7h35m12.713516183s) [seg_pos={0 0}, upload=7765]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 00:01:59.126519673 (7h35m12.713575884s) [upload=7765]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:59.126523730 (7h35m12.713579952s) [upload=7765]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:08.188656920 (7h35m21.775713142s) [upload=7765]: Done uploading segments... Aug 8 00:02:08.188678911 (7h35m21.775735132s) [upload=7765]: Upload finished. err: <nil> Aug 8 00:02:15.850573460 (7h35m29.437629681s) [upload=7774]: Starting upload Aug 8 00:02:15.850635545 (7h35m29.437691756s) [upload=7774]: Done starting upload Aug 8 00:02:15.850731674 (7h35m29.437787915s) [upload=7774]: Uploading segments... Aug 8 00:02:15.850867428 (7h35m29.437923649s) [upload=7774]: Got next segment. Inline: false Aug 8 00:02:15.850899257 (7h35m29.437955478s) [upload=7774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:15.850902814 (7h35m29.437959025s) [upload=7774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:15.831551466 (7h36m29.418607678s) [upload=7774]: Next segment err: context deadline exceeded Aug 8 00:03:15.831689523 (7h36m29.418745744s) [upload=7774]: Done uploading segments... Aug 8 00:03:15.831713769 (7h36m29.41876999s) [upload=7774]: Upload finished. err: context deadline exceeded Duration: 1.225103885s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.036726374s FAIL (2025-08-08 00:07:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 1.036749799s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [............................................XX.X.......X....] duration: 175.376035ms FAIL (2025-08-08 00:07:06 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:01.615214835 (7h30m15.202271047s) [upload=7695]: Starting upload Aug 7 23:57:01.615271901 (7h30m15.202328113s) [upload=7695]: Done starting upload Aug 7 23:57:01.615321205 (7h30m15.202377415s) [upload=7695]: Uploading segments... Aug 7 23:57:01.615349317 (7h30m15.202405538s) [upload=7695]: Got next segment. Inline: true Aug 7 23:57:01.615353044 (7h30m15.202409265s) [upload=7695]: Waiting for error group managing segments... Aug 7 23:57:01.615383740 (7h30m15.202439962s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:01.615387939 (7h30m15.20244416s) [upload=7695]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:57:01.615390112 (7h30m15.202446333s) [upload=7695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:01.690692823 (7h30m15.277749044s) [upload=7695]: Done uploading segments... Aug 7 23:57:01.690706398 (7h30m15.277762619s) [upload=7695]: Upload finished. err: <nil> Aug 7 23:58:00.120218851 (7h31m13.707275102s) [upload=7708]: Starting upload Aug 7 23:58:00.120294983 (7h31m13.707351225s) [upload=7708]: Done starting upload Aug 7 23:58:00.120367398 (7h31m13.707423649s) [upload=7708]: Uploading segments... Aug 7 23:58:00.120467996 (7h31m13.707524237s) [upload=7708]: Got next segment. Inline: true Aug 7 23:58:00.120477604 (7h31m13.707533845s) [upload=7708]: Waiting for error group managing segments... Aug 7 23:58:00.120533818 (7h31m13.70759006s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:00.120539920 (7h31m13.707596171s) [upload=7708]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:00.120543877 (7h31m13.707600119s) [upload=7708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:00.174307936 (7h31m13.761364207s) [upload=7708]: Done uploading segments... Aug 7 23:58:00.174326000 (7h31m13.761382251s) [upload=7708]: Upload finished. err: <nil> Aug 7 23:58:45.107350981 (7h31m58.694407202s) [upload=7720]: Starting upload Aug 7 23:58:45.107411425 (7h31m58.694467646s) [upload=7720]: Done starting upload Aug 7 23:58:45.107482426 (7h31m58.694538638s) [upload=7720]: Uploading segments... Aug 7 23:58:45.107519536 (7h31m58.694575747s) [upload=7720]: Got next segment. Inline: true Aug 7 23:58:45.107525497 (7h31m58.694581739s) [upload=7720]: Waiting for error group managing segments... Aug 7 23:58:45.107565932 (7h31m58.694622153s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:45.107570511 (7h31m58.694626722s) [upload=7720]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:45.107573006 (7h31m58.694629227s) [upload=7720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:45.172654338 (7h31m58.75971055s) [upload=7720]: Done uploading segments... Aug 7 23:58:45.172668515 (7h31m58.759724726s) [upload=7720]: Upload finished. err: <nil> Aug 7 23:59:21.615155283 (7h32m35.202211525s) [upload=7729]: Starting upload Aug 7 23:59:21.615232267 (7h32m35.202288508s) [upload=7729]: Done starting upload Aug 7 23:59:21.615296597 (7h32m35.202352849s) [upload=7729]: Uploading segments... Aug 7 23:59:21.615335539 (7h32m35.202391781s) [upload=7729]: Got next segment. Inline: true Aug 7 23:59:21.615340669 (7h32m35.202396911s) [upload=7729]: Waiting for error group managing segments... Aug 7 23:59:21.615385253 (7h32m35.202441494s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:21.615391084 (7h32m35.202447325s) [upload=7729]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:21.615394560 (7h32m35.202450801s) [upload=7729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:59:21.668842243 (7h32m35.255898484s) [upload=7729]: Done uploading segments... Aug 7 23:59:21.668861809 (7h32m35.255918051s) [upload=7729]: Upload finished. err: <nil> Aug 8 00:00:25.545668050 (7h33m39.132724271s) [upload=7746]: Starting upload Aug 8 00:00:25.545711883 (7h33m39.132768104s) [upload=7746]: Done starting upload Aug 8 00:00:25.545753380 (7h33m39.132809601s) [upload=7746]: Uploading segments... Aug 8 00:00:25.545779458 (7h33m39.132835669s) [upload=7746]: Got next segment. Inline: true Aug 8 00:00:25.545783446 (7h33m39.132839657s) [upload=7746]: Waiting for error group managing segments... Aug 8 00:00:25.545816808 (7h33m39.132873029s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:00:25.545820646 (7h33m39.132876857s) [upload=7746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:00:25.545823060 (7h33m39.132879281s) [upload=7746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:25.635202404 (7h33m39.222258625s) [upload=7746]: Done uploading segments... Aug 8 00:00:25.635216491 (7h33m39.222272702s) [upload=7746]: Upload finished. err: <nil> Aug 8 00:01:34.284952530 (7h34m47.872008771s) [upload=7764]: Starting upload Aug 8 00:01:34.285043029 (7h34m47.872099269s) [upload=7764]: Done starting upload Aug 8 00:01:34.285132245 (7h34m47.872188496s) [upload=7764]: Uploading segments... Aug 8 00:01:34.285212064 (7h34m47.872268435s) [upload=7764]: Got next segment. Inline: true Aug 8 00:01:34.285223345 (7h34m47.872279596s) [upload=7764]: Waiting for error group managing segments... Aug 8 00:01:34.285292926 (7h34m47.872349166s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:34.285299608 (7h34m47.872355849s) [upload=7764]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:01:34.285303245 (7h34m47.872359486s) [upload=7764]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:34.378710586 (7h34m47.965766827s) [upload=7764]: Done uploading segments... Aug 8 00:01:34.378730543 (7h34m47.965786785s) [upload=7764]: Upload finished. err: <nil> Aug 8 00:02:10.815209156 (7h35m24.402265377s) [upload=7772]: Starting upload Aug 8 00:02:10.815254480 (7h35m24.402310691s) [upload=7772]: Done starting upload Aug 8 00:02:10.815298863 (7h35m24.402355084s) [upload=7772]: Uploading segments... Aug 8 00:02:10.815336003 (7h35m24.402392224s) [upload=7772]: Got next segment. Inline: true Aug 8 00:02:10.815341934 (7h35m24.402398145s) [upload=7772]: Waiting for error group managing segments... Aug 8 00:02:10.815389482 (7h35m24.402445703s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:10.815395894 (7h35m24.402452115s) [upload=7772]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:10.815399912 (7h35m24.402456133s) [upload=7772]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:10.887039045 (7h35m24.474095256s) [upload=7772]: Done uploading segments... Aug 8 00:02:10.887052861 (7h35m24.474109072s) [upload=7772]: Upload finished. err: <nil> Aug 8 00:02:49.517222940 (7h36m3.104279182s) [upload=7784]: Starting upload Aug 8 00:02:49.517301727 (7h36m3.104357969s) [upload=7784]: Done starting upload Aug 8 00:02:49.517376386 (7h36m3.104432658s) [upload=7784]: Uploading segments... Aug 8 00:02:49.517523962 (7h36m3.104580243s) [upload=7784]: Got next segment. Inline: true Aug 8 00:02:49.517535033 (7h36m3.104591275s) [upload=7784]: Waiting for error group managing segments... Aug 8 00:02:49.517609392 (7h36m3.104665633s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:49.517619160 (7h36m3.104675401s) [upload=7784]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 00:02:49.517624850 (7h36m3.104681092s) [upload=7784]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:03:49.496721853 (7h37m3.083778104s) [upload=7784]: Done uploading segments... Aug 8 00:03:49.496758902 (7h37m3.083815144s) [upload=7784]: Upload finished. err: metaclient: context deadline exceeded Duration:* 175.397255ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [X.X......................................XXXXX.....X.X......] duration: 364.271954ms FAIL (2025-08-08 00:07:02 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== WBGOMPMH5AVKIWVIZUACOR7W6V6N44SOWF6EMHFDVSA, seg_pos={0 0}, upload=7782]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: rpc: dial tcp 104.36.82.15:4257: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:02:43.314543863 (7h35m56.901600154s) [seg_pos={0 0}, upload=7782]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:02:43.314657585 (7h35m56.901713837s) [upload=7782]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:02:43.314666051 (7h35m56.901722333s) [upload=7782]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:02:43.422269801 (7h35m57.009326042s) [upload=7782]: Done uploading segments... Aug 8 00:02:43.422289558 (7h35m57.009345809s) [upload=7782]: Upload finished. err: <nil> Aug 8 00:03:32.308703346 (7h36m45.895759557s) [upload=7795]: Starting upload Aug 8 00:03:32.308753630 (7h36m45.895809841s) [upload=7795]: Done starting upload Aug 8 00:03:32.308793925 (7h36m45.895850146s) [upload=7795]: Uploading segments... Aug 8 00:03:32.308975665 (7h36m45.896031886s) [upload=7795]: Got next segment. Inline: false Aug 8 00:03:32.309009528 (7h36m45.896065749s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:03:32.309013524 (7h36m45.896069745s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:03:58.580498109 (7h37m12.16755433s) [seg_pos={0 0}, upload=7795]: Begin upload segment... Aug 8 00:03:58.580538264 (7h37m12.167594485s) [seg_pos={0 0}, upload=7795]: Done begin upload segment. Aug 8 00:03:58.580559865 (7h37m12.167616086s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580574251 (7h37m12.167630462s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580869863 (7h37m12.167926084s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.580881295 (7h37m12.167937505s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581098260 (7h37m12.168154481s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581183999 (7h37m12.16824021s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581311307 (7h37m12.168367518s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581487295 (7h37m12.168543516s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581601057 (7h37m12.168657278s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.581752490 (7h37m12.168808711s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582228067 (7h37m12.169284288s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582244639 (7h37m12.169300849s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582526165 (7h37m12.169582386s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.582695160 (7h37m12.169751381s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583115203 (7h37m12.170171414s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.583728338 (7h37m12.170784549s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Uploading piece... Aug 8 00:03:58.667271549 (7h37m12.25432777s) [upload=7795]: Next returned nil segment Aug 8 00:03:58.667283711 (7h37m12.254339922s) [upload=7795]: Waiting for error group managing segments... Aug 8 00:03:58.677898323 (7h37m12.264954534s) [noise=true , address=45.91.116.12:4227 , piece_id=TF2QQW5CCDUBJ4UXYUCJZXKZDVLMBHIAMKTIBYZZVMMOC36UCKCQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.679486528 (7h37m12.266542739s) [noise=true , address=45.91.116.7:4010 , piece_id=6NZDG5MVZ3JUUJHRNYD6BXVOGP2YJGNFQFXHKO3QDFBQLVQAXGBQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683075307 (7h37m12.270131518s) [noise=true , address=77.90.31.19:4843 , piece_id=VAWTK3TV7CORLQKZ2F6SAWYWJX5KB7RZYCRGADAS6ZMVYEX5P4JQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683152210 (7h37m12.270208421s) [noise=true , address=77.90.31.12:4589 , piece_id=LK7IQUKIFR3MEC22YASIJ5CCELWBRCYAZNHTJIC24LKX7GID4YOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683440167 (7h37m12.270496388s) [noise=true , address=45.91.116.8:4045 , piece_id=K4VWENQAB5BP33H4GVZ3GNJWCFGTEOGEDPDANQOUYEZEHB22K5VA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.683855744 (7h37m12.270911944s) [noise=true , address=45.91.116.9:4262 , piece_id=HIKBSJRRDDMJ2G42AT5DAD67ODKSV3Y35JTIM3OCAEKB4HEMCP3Q, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.688501986 (7h37m12.275558197s) [noise=true , address=77.90.31.21:4878 , piece_id=CGXHPLOCRSUXTPQQ3QSB2P2Y7PSWBYXXXS6SOBKFR37VA2FAARNQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.694992971 (7h37m12.282049182s) [noise=true , address=45.91.116.4:4938 , piece_id=Z7QWDUM4SGPM2EMUWID7NGCQ5NCV6YZ5ITFSDGLPD4CYBPH7Q7HQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.697087700 (7h37m12.284143921s) [noise=true , address=104.36.82.16:4293 , piece_id=TJ47UI5MULFCQGPCJFJQQG7XDHRAO7WAIFTHKTX6KYKMV63FNH6A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.704286256 (7h37m12.291342468s) [noise=true , address=104.36.82.10:4077 , piece_id=C3ZUW7IY6PDUPVG426W45UNGOF46TLMAVUNBEOPKXPQM7IDIUOWA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.709672189 (7h37m12.2967284s) [noise=true , address=104.36.82.16:4294 , piece_id=ELKWKGM3IEIPFRVTCWRMBW5MLIRFOYWWPBEURBAYWX5RDKP6PVEQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.710566509 (7h37m12.297622719s) [noise=true , address=77.90.31.12:4592 , piece_id=ZWSEAW6XXQCVLAH2KLAXX2H4C4OJLLSJZNEKNGSQBYSC6DGCSTOQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732028516 (7h37m12.319084727s) [noise=true , address=104.36.82.2:4732 , piece_id=B3PIEKCUWKVC4BQP6VNCKY2DXYYGWEMJNOGIO6JOZBHPR52WEO7A, seg_pos={0 0}, upload=7795]: Done uploading piece. err: <nil> Aug 8 00:03:58.732043704 (7h37m12.319099915s) [seg_pos={0 0}, upload=7795]: Segment reached optimal threshold of 13 pieces. Aug 8 00:03:58.732159080 (7h37m12.319215291s) [noise=true , address=104.36.82.3:4767 , piece_id=RCJKFIH7VABX4VT5PN4HCJL65EYWIOCM7SELD7KDQB5GT6LTPFSA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: expected piece hash; context canceled; EOF Aug 8 00:03:58.732228479 (7h37m12.31928469s) [noise=true , address=104.36.82.14:4223 , piece_id=4CGK72BVESCYNYGG6O2HG6BI6XFCVFRBDAYR7VZPHAU3PAEV7JJQ, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: read tcp 5.161.89.252:32842->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732251893 (7h37m12.319308114s) [noise=true , address=104.36.82.15:4258 , piece_id=VXNYVGZIZMFIMQDD6F5WYQHDGWRRCXGAK5SA7U25PXC22XZ45GXA, seg_pos={0 0}, upload=7795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: read tcp 5.161.89.252:18008->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 00:03:58.732270468 (7h37m12.319326689s) [seg_pos={0 0}, upload=7795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 8 00:03:58.732342051 (7h37m12.319398262s) [upload=7795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:03:58.732345868 (7h37m12.319402089s) [upload=7795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:04:02.460668475 (7h37m16.047724696s) [upload=7795]: Done uploading segments... Aug 8 00:04:02.460695935 (7h37m16.047752156s) [upload=7795]: Upload finished. err: metaclient: context deadline exceeded Duration: 364.295498ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...................................XXXXXXXXX...X............] duration: 165.94393ms FAIL (2025-08-08 00:06:59 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 165.966653ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X...........X.......] duration: 481.5976ms FAIL (2025-08-08 00:06:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 481.61864ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)