Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXX............................] duration: 203.42963ms FAIL (2025-08-07 22:09:43 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== shed. err: metaclient: context deadline exceeded Aug 7 21:38:26.513648977 (5h11m40.100705218s) [upload=5344]: Starting upload Aug 7 21:38:26.513731310 (5h11m40.100787552s) [upload=5344]: Done starting upload Aug 7 21:38:26.513792245 (5h11m40.100848496s) [upload=5344]: Uploading segments... Aug 7 21:38:26.513834112 (5h11m40.100890354s) [upload=5344]: Got next segment. Inline: true Aug 7 21:38:26.513839653 (5h11m40.100895904s) [upload=5344]: Waiting for error group managing segments... Aug 7 21:38:26.513884006 (5h11m40.100940247s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:26.513889947 (5h11m40.100946188s) [upload=5344]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:26.513893434 (5h11m40.100949675s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:56.502052245 (5h12m10.089108537s) [upload=5344]: Done uploading segments... Aug 7 21:38:56.502102730 (5h12m10.089158972s) [upload=5344]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:24.008006172 (5h12m37.595062394s) [upload=5359]: Starting upload Aug 7 21:39:24.008059211 (5h12m37.595115422s) [upload=5359]: Done starting upload Aug 7 21:39:24.008103604 (5h12m37.595159835s) [upload=5359]: Uploading segments... Aug 7 21:39:24.008156191 (5h12m37.595212413s) [upload=5359]: Got next segment. Inline: true Aug 7 21:39:24.008162193 (5h12m37.595218414s) [upload=5359]: Waiting for error group managing segments... Aug 7 21:39:24.008207327 (5h12m37.595263548s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:24.008212668 (5h12m37.595268878s) [upload=5359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:24.008216064 (5h12m37.595272285s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:53.997226503 (5h13m7.584282744s) [upload=5359]: Done uploading segments... Aug 7 21:39:53.997270334 (5h13m7.584326586s) [upload=5359]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:36.779619947 (5h13m50.366676188s) [upload=5383]: Starting upload Aug 7 21:40:36.779696139 (5h13m50.36675238s) [upload=5383]: Done starting upload Aug 7 21:40:36.779752775 (5h13m50.366809016s) [upload=5383]: Uploading segments... Aug 7 21:40:36.779788803 (5h13m50.366845044s) [upload=5383]: Got next segment. Inline: true Aug 7 21:40:36.779794423 (5h13m50.366850655s) [upload=5383]: Waiting for error group managing segments... Aug 7 21:40:36.779835950 (5h13m50.366892191s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:36.779841010 (5h13m50.366897241s) [upload=5383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:36.779844195 (5h13m50.366900437s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:06.765292852 (5h14m20.352349064s) [upload=5383]: Done uploading segments... Aug 7 21:41:06.765314052 (5h14m20.352370273s) [upload=5383]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:47.961322510 (5h15m1.548378802s) [upload=5399]: Starting upload Aug 7 21:41:47.961442915 (5h15m1.548499166s) [upload=5399]: Done starting upload Aug 7 21:41:47.961510201 (5h15m1.548566443s) [upload=5399]: Uploading segments... Aug 7 21:41:47.961591062 (5h15m1.548647313s) [upload=5399]: Got next segment. Inline: true Aug 7 21:41:47.961599218 (5h15m1.548655458s) [upload=5399]: Waiting for error group managing segments... Aug 7 21:41:47.961647759 (5h15m1.548703999s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:47.961653760 (5h15m1.548710001s) [upload=5399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:47.961657546 (5h15m1.548713787s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:17.948677724 (5h15m31.535733945s) [upload=5399]: Done uploading segments... Aug 7 21:42:17.948691099 (5h15m31.53574731s) [upload=5399]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.081006705 (5h15m31.668062915s) [upload=5405]: Starting upload Aug 7 21:42:18.081065655 (5h15m31.668121866s) [upload=5405]: Done starting upload Aug 7 21:42:18.081113425 (5h15m31.668169635s) [upload=5405]: Uploading segments... Aug 7 21:42:18.081164870 (5h15m31.668221091s) [upload=5405]: Got next segment. Inline: true Aug 7 21:42:18.081169459 (5h15m31.66822568s) [upload=5405]: Waiting for error group managing segments... Aug 7 21:42:18.081201809 (5h15m31.66825803s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.081205546 (5h15m31.668261757s) [upload=5405]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.081208271 (5h15m31.668264483s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:18.161269758 (5h15m31.748325999s) [upload=5405]: Done uploading segments... Aug 7 21:42:18.161287411 (5h15m31.748343652s) [upload=5405]: Upload finished. err: <nil> Aug 7 21:43:12.878567983 (5h16m26.465624204s) [upload=5420]: Starting upload Aug 7 21:43:12.878678820 (5h16m26.465735041s) [upload=5420]: Done starting upload Aug 7 21:43:12.878731839 (5h16m26.46578806s) [upload=5420]: Uploading segments... Aug 7 21:43:12.878762908 (5h16m26.465819129s) [upload=5420]: Got next segment. Inline: true Aug 7 21:43:12.878766635 (5h16m26.465822856s) [upload=5420]: Waiting for error group managing segments... Aug 7 21:43:12.878799204 (5h16m26.465855425s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:12.878803332 (5h16m26.465859543s) [upload=5420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:12.878805566 (5h16m26.465861787s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:42.863779226 (5h16m56.450835478s) [upload=5420]: Done uploading segments... Aug 7 21:43:42.863802450 (5h16m56.450858691s) [upload=5420]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:51.464096058 (5h17m5.051152299s) [upload=5431]: Starting upload Aug 7 21:43:51.464210432 (5h17m5.051266673s) [upload=5431]: Done starting upload Aug 7 21:43:51.464274041 (5h17m5.051330282s) [upload=5431]: Uploading segments... Aug 7 21:43:51.464330065 (5h17m5.051386366s) [upload=5431]: Got next segment. Inline: true Aug 7 21:43:51.464343630 (5h17m5.051399912s) [upload=5431]: Waiting for error group managing segments... Aug 7 21:43:51.464415315 (5h17m5.051471556s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:51.464424673 (5h17m5.051480923s) [upload=5431]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:51.464431204 (5h17m5.051487455s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:21.448488824 (5h17m35.035545036s) [upload=5431]: Done uploading segments... Aug 7 21:44:21.448528209 (5h17m35.03558443s) [upload=5431]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:50.396194570 (5h18m3.983250782s) [upload=5444]: Starting upload Aug 7 21:44:50.396243653 (5h18m3.983299864s) [upload=5444]: Done starting upload Aug 7 21:44:50.396286463 (5h18m3.983342684s) [upload=5444]: Uploading segments... Aug 7 21:44:50.396309054 (5h18m3.983365266s) [upload=5444]: Got next segment. Inline: true Aug 7 21:44:50.396312381 (5h18m3.983368592s) [upload=5444]: Waiting for error group managing segments... Aug 7 21:44:50.396342687 (5h18m3.983398908s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:50.396346635 (5h18m3.983402856s) [upload=5444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:50.396348889 (5h18m3.98340511s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:20.384461417 (5h18m33.971517668s) [upload=5444]: Done uploading segments... Aug 7 21:45:20.384482547 (5h18m33.971538788s) [upload=5444]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:58.414522311 (5h19m12.001578522s) [upload=5465]: Starting upload Aug 7 21:45:58.414582292 (5h19m12.001638504s) [upload=5465]: Done starting upload Aug 7 21:45:58.414634661 (5h19m12.001690882s) [upload=5465]: Uploading segments... Aug 7 21:45:58.414662251 (5h19m12.001718462s) [upload=5465]: Got next segment. Inline: true Aug 7 21:45:58.414665918 (5h19m12.00172213s) [upload=5465]: Waiting for error group managing segments... Aug 7 21:45:58.414697247 (5h19m12.001753468s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.414700753 (5h19m12.001756975s) [upload=5465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:58.414703007 (5h19m12.001759229s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:28.397843251 (5h19m41.984899492s) [upload=5465]: Done uploading segments... Aug 7 21:46:28.397885870 (5h19m41.984942112s) [upload=5465]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:07.964355881 (5h20m21.551412133s) [upload=5484]: Starting upload Aug 7 21:47:07.964442332 (5h20m21.551498584s) [upload=5484]: Done starting upload Aug 7 21:47:07.964529184 (5h20m21.551585426s) [upload=5484]: Uploading segments... Aug 7 21:47:07.964568658 (5h20m21.5516249s) [upload=5484]: Got next segment. Inline: true Aug 7 21:47:07.964573968 (5h20m21.551630209s) [upload=5484]: Waiting for error group managing segments... Aug 7 21:47:07.964617630 (5h20m21.551673881s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:07.964623640 (5h20m21.551679882s) [upload=5484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:07.964628048 (5h20m21.55168429s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:37.944795042 (5h20m51.531851283s) [upload=5484]: Done uploading segments... Aug 7 21:47:37.944820539 (5h20m51.531876781s) [upload=5484]: Upload finished. err: metaclient: context deadline exceeded Duration: 203.461499ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [..........X...X...X.........................................] duration: 131.862019ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:09:35 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 131.97593ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXX.X.................................] duration: 179.769092ms FAIL (2025-08-07 22:09:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 179.793388ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXX.XX......................................] duration: 514.441856ms FAIL (2025-08-07 22:09:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 514.460271ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [...XX...X...................................................] duration: 23.738291ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:09:16 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 23.763408ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXX.X.XX............................] duration: 164.446251ms FAIL (2025-08-07 22:09:13 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 31725988s) [upload=5324]: Upload finished. err: <nil> Aug 7 21:38:15.287715124 (5h11m28.874771365s) [upload=5341]: Starting upload Aug 7 21:38:15.287780276 (5h11m28.874836516s) [upload=5341]: Done starting upload Aug 7 21:38:15.287844856 (5h11m28.874901107s) [upload=5341]: Uploading segments... Aug 7 21:38:15.287889961 (5h11m28.874946201s) [upload=5341]: Got next segment. Inline: true Aug 7 21:38:15.287895600 (5h11m28.874951842s) [upload=5341]: Waiting for error group managing segments... Aug 7 21:38:15.287984066 (5h11m28.875040317s) [upload=5341]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:15.287990077 (5h11m28.875046318s) [upload=5341]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:15.287993564 (5h11m28.875049804s) [upload=5341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:15.275679218 (5h12m28.862735459s) [upload=5341]: Done uploading segments... Aug 7 21:39:15.275708622 (5h12m28.862764883s) [upload=5341]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:15.358605080 (5h12m28.945661322s) [upload=5358]: Starting upload Aug 7 21:39:15.358690641 (5h12m28.945746882s) [upload=5358]: Done starting upload Aug 7 21:39:15.358748869 (5h12m28.945805111s) [upload=5358]: Uploading segments... Aug 7 21:39:15.358790938 (5h12m28.94584718s) [upload=5358]: Got next segment. Inline: true Aug 7 21:39:15.358796779 (5h12m28.94585302s) [upload=5358]: Waiting for error group managing segments... Aug 7 21:39:15.358840480 (5h12m28.945896721s) [upload=5358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:15.358846441 (5h12m28.945902683s) [upload=5358]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:15.358849918 (5h12m28.945906159s) [upload=5358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:15.344303433 (5h13m28.931359654s) [upload=5358]: Done uploading segments... Aug 7 21:40:15.344323120 (5h13m28.931379331s) [upload=5358]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:15.438334176 (5h13m29.025390397s) [upload=5373]: Starting upload Aug 7 21:40:15.438379379 (5h13m29.025435601s) [upload=5373]: Done starting upload Aug 7 21:40:15.438418503 (5h13m29.025474714s) [upload=5373]: Uploading segments... Aug 7 21:40:15.438447206 (5h13m29.025503427s) [upload=5373]: Got next segment. Inline: true Aug 7 21:40:15.438450683 (5h13m29.025506894s) [upload=5373]: Waiting for error group managing segments... Aug 7 21:40:15.438481920 (5h13m29.025538141s) [upload=5373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:15.438487511 (5h13m29.025543722s) [upload=5373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:15.438489826 (5h13m29.025546047s) [upload=5373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:15.423387101 (5h14m29.010443342s) [upload=5373]: Done uploading segments... Aug 7 21:41:15.423408011 (5h14m29.010464251s) [upload=5373]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:15.514709386 (5h14m29.101765628s) [upload=5390]: Starting upload Aug 7 21:41:15.514790870 (5h14m29.10184711s) [upload=5390]: Done starting upload Aug 7 21:41:15.514852845 (5h14m29.101909086s) [upload=5390]: Uploading segments... Aug 7 21:41:15.514892409 (5h14m29.10194865s) [upload=5390]: Got next segment. Inline: true Aug 7 21:41:15.514898240 (5h14m29.101954481s) [upload=5390]: Waiting for error group managing segments... Aug 7 21:41:15.514941951 (5h14m29.101998192s) [upload=5390]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:15.514947551 (5h14m29.102003793s) [upload=5390]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:15.514951138 (5h14m29.102007389s) [upload=5390]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:15.497042181 (5h15m29.084098402s) [upload=5390]: Done uploading segments... Aug 7 21:42:15.497058301 (5h15m29.084114512s) [upload=5390]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:15.596278951 (5h15m29.183335172s) [upload=5404]: Starting upload Aug 7 21:42:15.596337329 (5h15m29.183393541s) [upload=5404]: Done starting upload Aug 7 21:42:15.596382114 (5h15m29.183438324s) [upload=5404]: Uploading segments... Aug 7 21:42:15.596411919 (5h15m29.18346814s) [upload=5404]: Got next segment. Inline: true Aug 7 21:42:15.596415937 (5h15m29.183472158s) [upload=5404]: Waiting for error group managing segments... Aug 7 21:42:15.596447405 (5h15m29.183503626s) [upload=5404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:15.596451042 (5h15m29.183507263s) [upload=5404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:15.596453266 (5h15m29.183509487s) [upload=5404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:15.584735444 (5h16m29.171791655s) [upload=5404]: Done uploading segments... Aug 7 21:43:15.584747787 (5h16m29.171804008s) [upload=5404]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:22.691047415 (5h16m36.278103646s) [upload=5423]: Starting upload Aug 7 21:43:22.691110724 (5h16m36.278166955s) [upload=5423]: Done starting upload Aug 7 21:43:22.691177108 (5h16m36.278233338s) [upload=5423]: Uploading segments... Aug 7 21:43:22.691210019 (5h16m36.27826625s) [upload=5423]: Got next segment. Inline: true Aug 7 21:43:22.691214187 (5h16m36.278270418s) [upload=5423]: Waiting for error group managing segments... Aug 7 21:43:22.691250704 (5h16m36.278306925s) [upload=5423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:22.691255263 (5h16m36.278311494s) [upload=5423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:22.691258188 (5h16m36.278314419s) [upload=5423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:22.751511620 (5h16m36.338567872s) [upload=5423]: Done uploading segments... Aug 7 21:43:22.751530204 (5h16m36.338586455s) [upload=5423]: Upload finished. err: <nil> Aug 7 21:44:13.185700222 (5h17m26.772756433s) [upload=5437]: Starting upload Aug 7 21:44:13.185766515 (5h17m26.772822736s) [upload=5437]: Done starting upload Aug 7 21:44:13.185811771 (5h17m26.772867981s) [upload=5437]: Uploading segments... Aug 7 21:44:13.185877192 (5h17m26.772933443s) [upload=5437]: Got next segment. Inline: true Aug 7 21:44:13.185884005 (5h17m26.772940226s) [upload=5437]: Waiting for error group managing segments... Aug 7 21:44:13.185926104 (5h17m26.772982314s) [upload=5437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:13.185931043 (5h17m26.772987254s) [upload=5437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:13.185934560 (5h17m26.77299077s) [upload=5437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:13.172887995 (5h18m26.759944236s) [upload=5437]: Done uploading segments... Aug 7 21:45:13.172908133 (5h18m26.759964374s) [upload=5437]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:13.256441576 (5h18m26.843497827s) [upload=5449]: Starting upload Aug 7 21:45:13.256514653 (5h18m26.843570894s) [upload=5449]: Done starting upload Aug 7 21:45:13.256603699 (5h18m26.84365995s) [upload=5449]: Uploading segments... Aug 7 21:45:13.256644835 (5h18m26.843701076s) [upload=5449]: Got next segment. Inline: true Aug 7 21:45:13.256650195 (5h18m26.843706436s) [upload=5449]: Waiting for error group managing segments... Aug 7 21:45:13.256695270 (5h18m26.843751511s) [upload=5449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:13.256701020 (5h18m26.843757262s) [upload=5449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:13.256704657 (5h18m26.843760898s) [upload=5449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:13.340684827 (5h18m26.927741079s) [upload=5449]: Done uploading segments... Aug 7 21:45:13.340709023 (5h18m26.927765264s) [upload=5449]: Upload finished. err: <nil> Aug 7 21:46:18.644805531 (5h19m32.231861771s) [upload=5472]: Starting upload Aug 7 21:46:18.644902892 (5h19m32.231959133s) [upload=5472]: Done starting upload Aug 7 21:46:18.644968324 (5h19m32.232024565s) [upload=5472]: Uploading segments... Aug 7 21:46:18.645015051 (5h19m32.232071292s) [upload=5472]: Got next segment. Inline: true Aug 7 21:46:18.645020682 (5h19m32.232076933s) [upload=5472]: Waiting for error group managing segments... Aug 7 21:46:18.645064344 (5h19m32.232120585s) [upload=5472]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:18.645070025 (5h19m32.232126266s) [upload=5472]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:18.645073771 (5h19m32.232130013s) [upload=5472]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:18.630338668 (5h20m32.217394879s) [upload=5472]: Done uploading segments... Aug 7 21:47:18.630353927 (5h20m32.217410137s) [upload=5472]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:18.710984265 (5h20m32.298040506s) [upload=5489]: Starting upload Aug 7 21:47:18.711056571 (5h20m32.298112812s) [upload=5489]: Done starting upload Aug 7 21:47:18.711125208 (5h20m32.29818145s) [upload=5489]: Uploading segments... Aug 7 21:47:18.711173729 (5h20m32.29822996s) [upload=5489]: Got next segment. Inline: true Aug 7 21:47:18.711179229 (5h20m32.298235471s) [upload=5489]: Waiting for error group managing segments... Aug 7 21:47:18.711220847 (5h20m32.298277087s) [upload=5489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:18.711225816 (5h20m32.298282057s) [upload=5489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:18.711229112 (5h20m32.298285354s) [upload=5489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:18.698417946 (5h21m32.285474187s) [upload=5489]: Done uploading segments... Aug 7 21:48:18.698439776 (5h21m32.285496017s) [upload=5489]: Upload finished. err: metaclient: context deadline exceeded Duration: 164.468653ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXX..X.....................................] duration: 388.265861ms FAIL (2025-08-07 22:09:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 (5h14m28.062269058s) [upload=5388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:25.266685321 (5h14m38.853741562s) [upload=5388]: Done uploading segments... Aug 7 21:41:25.266706391 (5h14m38.853762631s) [upload=5388]: Upload finished. err: <nil> Aug 7 21:41:41.070176076 (5h14m54.657232316s) [upload=5396]: Starting upload Aug 7 21:41:41.070246557 (5h14m54.657302798s) [upload=5396]: Done starting upload Aug 7 21:41:41.070459143 (5h14m54.657515385s) [upload=5396]: Uploading segments... Aug 7 21:41:41.070697238 (5h14m54.657753479s) [upload=5396]: Got next segment. Inline: false Aug 7 21:41:41.070732264 (5h14m54.657788505s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:41.070737293 (5h14m54.657793524s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:41:41.825834701 (5h14m55.412890922s) [seg_pos={0 0}, upload=5396]: Begin upload segment... Aug 7 21:41:41.825865027 (5h14m55.412921237s) [seg_pos={0 0}, upload=5396]: Done begin upload segment. Aug 7 21:41:41.825886015 (5h14m55.412942237s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.825988818 (5h14m55.413045068s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826203979 (5h14m55.413260201s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826435742 (5h14m55.413491953s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826541530 (5h14m55.41359777s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826653037 (5h14m55.413709248s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826875332 (5h14m55.413931543s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826921499 (5h14m55.41397774s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827091606 (5h14m55.414147827s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827323819 (5h14m55.41438003s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827393500 (5h14m55.414449742s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827542938 (5h14m55.414599159s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827778078 (5h14m55.414834318s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827801491 (5h14m55.414857712s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828365735 (5h14m55.415421976s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828913136 (5h14m55.415969378s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.934607129 (5h14m55.52166334s) [upload=5396]: Next returned nil segment Aug 7 21:41:41.934619282 (5h14m55.521675503s) [upload=5396]: Waiting for error group managing segments... Aug 7 21:41:41.945118551 (5h14m55.532174772s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.945901252 (5h14m55.532957473s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946161258 (5h14m55.533217498s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946386648 (5h14m55.533442869s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949024992 (5h14m55.536081243s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949218724 (5h14m55.536274945s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949657033 (5h14m55.536713244s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949683371 (5h14m55.536739612s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.950070354 (5h14m55.537126575s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.953623165 (5h14m55.540679407s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.957708261 (5h14m55.544764482s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964243811 (5h14m55.551300021s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964842018 (5h14m55.551898239s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964860783 (5h14m55.551917004s) [seg_pos={0 0}, upload=5396]: Segment reached optimal threshold of 13 pieces. Aug 7 21:41:41.964962292 (5h14m55.552018503s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965019739 (5h14m55.55207595s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965087696 (5h14m55.552143947s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965118783 (5h14m55.552175005s) [seg_pos={0 0}, upload=5396]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:41:41.965239138 (5h14m55.552295349s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:41:41.965243687 (5h14m55.552299898s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:46.509965896 (5h15m0.097022118s) [upload=5396]: Done uploading segments... Aug 7 21:41:46.510002955 (5h15m0.097059167s) [upload=5396]: Upload finished. err: <nil> Aug 7 21:42:01.196715658 (5h15m14.783771899s) [upload=5401]: Starting upload Aug 7 21:42:01.196842946 (5h15m14.783899187s) [upload=5401]: Done starting upload Aug 7 21:42:01.196915852 (5h15m14.783972103s) [upload=5401]: Uploading segments... Aug 7 21:42:01.197283067 (5h15m14.784339309s) [upload=5401]: Got next segment. Inline: false Aug 7 21:42:01.197355652 (5h15m14.784411984s) [upload=5401]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:01.197365230 (5h15m14.784421472s) [upload=5401]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:42:31.170606605 (5h15m44.757662826s) [upload=5401]: Next segment err: context deadline exceeded Aug 7 21:42:31.170727651 (5h15m44.757783872s) [upload=5401]: Done uploading segments... Aug 7 21:42:31.170749551 (5h15m44.757805773s) [upload=5401]: Upload finished. err: context deadline exceeded Duration:* 388.293253ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXX..................................] duration: 1.730891768s FAIL (2025-08-07 22:09:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.730912095s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 203.075078ms FAIL (2025-08-07 22:09:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 203.094665ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 32 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXX...........................] duration: 462.600399ms FAIL (2025-08-07 22:09:04 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== shed. err: metaclient: context deadline exceeded Aug 7 21:38:26.513648977 (5h11m40.100705218s) [upload=5344]: Starting upload Aug 7 21:38:26.513731310 (5h11m40.100787552s) [upload=5344]: Done starting upload Aug 7 21:38:26.513792245 (5h11m40.100848496s) [upload=5344]: Uploading segments... Aug 7 21:38:26.513834112 (5h11m40.100890354s) [upload=5344]: Got next segment. Inline: true Aug 7 21:38:26.513839653 (5h11m40.100895904s) [upload=5344]: Waiting for error group managing segments... Aug 7 21:38:26.513884006 (5h11m40.100940247s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:26.513889947 (5h11m40.100946188s) [upload=5344]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:26.513893434 (5h11m40.100949675s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:56.502052245 (5h12m10.089108537s) [upload=5344]: Done uploading segments... Aug 7 21:38:56.502102730 (5h12m10.089158972s) [upload=5344]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:24.008006172 (5h12m37.595062394s) [upload=5359]: Starting upload Aug 7 21:39:24.008059211 (5h12m37.595115422s) [upload=5359]: Done starting upload Aug 7 21:39:24.008103604 (5h12m37.595159835s) [upload=5359]: Uploading segments... Aug 7 21:39:24.008156191 (5h12m37.595212413s) [upload=5359]: Got next segment. Inline: true Aug 7 21:39:24.008162193 (5h12m37.595218414s) [upload=5359]: Waiting for error group managing segments... Aug 7 21:39:24.008207327 (5h12m37.595263548s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:24.008212668 (5h12m37.595268878s) [upload=5359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:24.008216064 (5h12m37.595272285s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:53.997226503 (5h13m7.584282744s) [upload=5359]: Done uploading segments... Aug 7 21:39:53.997270334 (5h13m7.584326586s) [upload=5359]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:36.779619947 (5h13m50.366676188s) [upload=5383]: Starting upload Aug 7 21:40:36.779696139 (5h13m50.36675238s) [upload=5383]: Done starting upload Aug 7 21:40:36.779752775 (5h13m50.366809016s) [upload=5383]: Uploading segments... Aug 7 21:40:36.779788803 (5h13m50.366845044s) [upload=5383]: Got next segment. Inline: true Aug 7 21:40:36.779794423 (5h13m50.366850655s) [upload=5383]: Waiting for error group managing segments... Aug 7 21:40:36.779835950 (5h13m50.366892191s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:36.779841010 (5h13m50.366897241s) [upload=5383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:36.779844195 (5h13m50.366900437s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:06.765292852 (5h14m20.352349064s) [upload=5383]: Done uploading segments... Aug 7 21:41:06.765314052 (5h14m20.352370273s) [upload=5383]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:47.961322510 (5h15m1.548378802s) [upload=5399]: Starting upload Aug 7 21:41:47.961442915 (5h15m1.548499166s) [upload=5399]: Done starting upload Aug 7 21:41:47.961510201 (5h15m1.548566443s) [upload=5399]: Uploading segments... Aug 7 21:41:47.961591062 (5h15m1.548647313s) [upload=5399]: Got next segment. Inline: true Aug 7 21:41:47.961599218 (5h15m1.548655458s) [upload=5399]: Waiting for error group managing segments... Aug 7 21:41:47.961647759 (5h15m1.548703999s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:47.961653760 (5h15m1.548710001s) [upload=5399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:47.961657546 (5h15m1.548713787s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:17.948677724 (5h15m31.535733945s) [upload=5399]: Done uploading segments... Aug 7 21:42:17.948691099 (5h15m31.53574731s) [upload=5399]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.081006705 (5h15m31.668062915s) [upload=5405]: Starting upload Aug 7 21:42:18.081065655 (5h15m31.668121866s) [upload=5405]: Done starting upload Aug 7 21:42:18.081113425 (5h15m31.668169635s) [upload=5405]: Uploading segments... Aug 7 21:42:18.081164870 (5h15m31.668221091s) [upload=5405]: Got next segment. Inline: true Aug 7 21:42:18.081169459 (5h15m31.66822568s) [upload=5405]: Waiting for error group managing segments... Aug 7 21:42:18.081201809 (5h15m31.66825803s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.081205546 (5h15m31.668261757s) [upload=5405]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.081208271 (5h15m31.668264483s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:18.161269758 (5h15m31.748325999s) [upload=5405]: Done uploading segments... Aug 7 21:42:18.161287411 (5h15m31.748343652s) [upload=5405]: Upload finished. err: <nil> Aug 7 21:43:12.878567983 (5h16m26.465624204s) [upload=5420]: Starting upload Aug 7 21:43:12.878678820 (5h16m26.465735041s) [upload=5420]: Done starting upload Aug 7 21:43:12.878731839 (5h16m26.46578806s) [upload=5420]: Uploading segments... Aug 7 21:43:12.878762908 (5h16m26.465819129s) [upload=5420]: Got next segment. Inline: true Aug 7 21:43:12.878766635 (5h16m26.465822856s) [upload=5420]: Waiting for error group managing segments... Aug 7 21:43:12.878799204 (5h16m26.465855425s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:12.878803332 (5h16m26.465859543s) [upload=5420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:12.878805566 (5h16m26.465861787s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:42.863779226 (5h16m56.450835478s) [upload=5420]: Done uploading segments... Aug 7 21:43:42.863802450 (5h16m56.450858691s) [upload=5420]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:51.464096058 (5h17m5.051152299s) [upload=5431]: Starting upload Aug 7 21:43:51.464210432 (5h17m5.051266673s) [upload=5431]: Done starting upload Aug 7 21:43:51.464274041 (5h17m5.051330282s) [upload=5431]: Uploading segments... Aug 7 21:43:51.464330065 (5h17m5.051386366s) [upload=5431]: Got next segment. Inline: true Aug 7 21:43:51.464343630 (5h17m5.051399912s) [upload=5431]: Waiting for error group managing segments... Aug 7 21:43:51.464415315 (5h17m5.051471556s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:51.464424673 (5h17m5.051480923s) [upload=5431]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:51.464431204 (5h17m5.051487455s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:21.448488824 (5h17m35.035545036s) [upload=5431]: Done uploading segments... Aug 7 21:44:21.448528209 (5h17m35.03558443s) [upload=5431]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:50.396194570 (5h18m3.983250782s) [upload=5444]: Starting upload Aug 7 21:44:50.396243653 (5h18m3.983299864s) [upload=5444]: Done starting upload Aug 7 21:44:50.396286463 (5h18m3.983342684s) [upload=5444]: Uploading segments... Aug 7 21:44:50.396309054 (5h18m3.983365266s) [upload=5444]: Got next segment. Inline: true Aug 7 21:44:50.396312381 (5h18m3.983368592s) [upload=5444]: Waiting for error group managing segments... Aug 7 21:44:50.396342687 (5h18m3.983398908s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:50.396346635 (5h18m3.983402856s) [upload=5444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:50.396348889 (5h18m3.98340511s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:20.384461417 (5h18m33.971517668s) [upload=5444]: Done uploading segments... Aug 7 21:45:20.384482547 (5h18m33.971538788s) [upload=5444]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:58.414522311 (5h19m12.001578522s) [upload=5465]: Starting upload Aug 7 21:45:58.414582292 (5h19m12.001638504s) [upload=5465]: Done starting upload Aug 7 21:45:58.414634661 (5h19m12.001690882s) [upload=5465]: Uploading segments... Aug 7 21:45:58.414662251 (5h19m12.001718462s) [upload=5465]: Got next segment. Inline: true Aug 7 21:45:58.414665918 (5h19m12.00172213s) [upload=5465]: Waiting for error group managing segments... Aug 7 21:45:58.414697247 (5h19m12.001753468s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.414700753 (5h19m12.001756975s) [upload=5465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:58.414703007 (5h19m12.001759229s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:28.397843251 (5h19m41.984899492s) [upload=5465]: Done uploading segments... Aug 7 21:46:28.397885870 (5h19m41.984942112s) [upload=5465]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:07.964355881 (5h20m21.551412133s) [upload=5484]: Starting upload Aug 7 21:47:07.964442332 (5h20m21.551498584s) [upload=5484]: Done starting upload Aug 7 21:47:07.964529184 (5h20m21.551585426s) [upload=5484]: Uploading segments... Aug 7 21:47:07.964568658 (5h20m21.5516249s) [upload=5484]: Got next segment. Inline: true Aug 7 21:47:07.964573968 (5h20m21.551630209s) [upload=5484]: Waiting for error group managing segments... Aug 7 21:47:07.964617630 (5h20m21.551673881s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:07.964623640 (5h20m21.551679882s) [upload=5484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:07.964628048 (5h20m21.55168429s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:37.944795042 (5h20m51.531851283s) [upload=5484]: Done uploading segments... Aug 7 21:47:37.944820539 (5h20m51.531876781s) [upload=5484]: Upload finished. err: metaclient: context deadline exceeded Duration: 462.623151ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 20 out of 60 is failed (threshold=3) [.....XXXXXXXXXXXXXXXXX.XX......X............................] duration: 860.054477ms FAIL (2025-08-07 22:09:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== .251202997s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:13.664150523 (5h20m27.251206734s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:57.268638109 (5h21m10.855694349s) [seg_pos={0 0}, upload=5487]: Begin upload segment... Aug 7 21:47:57.268679396 (5h21m10.855735637s) [seg_pos={0 0}, upload=5487]: Done begin upload segment. Aug 7 21:47:57.268708491 (5h21m10.855764731s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.268768562 (5h21m10.855824803s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269205608 (5h21m10.856261849s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269223562 (5h21m10.856279802s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269609953 (5h21m10.856666204s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269616095 (5h21m10.856672345s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269967370 (5h21m10.857023612s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269989231 (5h21m10.857045472s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270389678 (5h21m10.85744593s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270397674 (5h21m10.857453915s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270761794 (5h21m10.857818034s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270773264 (5h21m10.857829505s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271108780 (5h21m10.858165022s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271123869 (5h21m10.85818011s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271518095 (5h21m10.858574346s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272004884 (5h21m10.859061125s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272493695 (5h21m10.859549937s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.273356537 (5h21m10.860412828s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.396251337 (5h21m10.983307579s) [upload=5487]: Next returned nil segment Aug 7 21:47:57.396270474 (5h21m10.983326714s) [upload=5487]: Waiting for error group managing segments... Aug 7 21:47:57.416691513 (5h21m11.003747763s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.420507917 (5h21m11.007564157s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.425516195 (5h21m11.012572446s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.427782495 (5h21m11.014838736s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.431475158 (5h21m11.018531398s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.432679355 (5h21m11.019735596s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.449481709 (5h21m11.036537951s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.451301637 (5h21m11.038357878s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.469893221 (5h21m11.056949472s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.470298257 (5h21m11.057354497s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.475204656 (5h21m11.062260917s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.549461095 (5h21m11.136517316s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.661813247 (5h21m11.248869498s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755309651 (5h21m11.342365892s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755326954 (5h21m11.342383194s) [seg_pos={0 0}, upload=5487]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:57.755590967 (5h21m11.342647207s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755687266 (5h21m11.342743507s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUV64TP8cTQhBjswnhcWfQHwYEEWqr5ASuPjtEx3X6ZQNVhS5): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755708706 (5h21m11.342764947s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection Aug 7 21:47:57.755820425 (5h21m11.342876656s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WKWWwa3svATpoNY1z7LNvYmA8gA6ZFpJsk17G4c37tDJVJ2wj): protocol: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection Aug 7 21:47:57.755837257 (5h21m11.342893498s) [seg_pos={0 0}, upload=5487]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:57.755901476 (5h21m11.342957717s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:57.755906926 (5h21m11.342963168s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:45.449359965 (5h21m59.036416176s) [upload=5487]: Done uploading segments... Aug 7 21:48:45.449390482 (5h21m59.036446703s) [upload=5487]: Upload finished. err: metaclient: context deadline exceeded Duration: 860.073965ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXX...X.....X............................] duration: 234.002508ms FAIL (2025-08-07 22:08:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 234.022697ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXX.X................................] duration: 180.37754ms FAIL (2025-08-07 22:08:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 180.399641ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXX..X....................................] duration: 557.664747ms FAIL (2025-08-07 22:08:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 (5h14m28.062269058s) [upload=5388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:25.266685321 (5h14m38.853741562s) [upload=5388]: Done uploading segments... Aug 7 21:41:25.266706391 (5h14m38.853762631s) [upload=5388]: Upload finished. err: <nil> Aug 7 21:41:41.070176076 (5h14m54.657232316s) [upload=5396]: Starting upload Aug 7 21:41:41.070246557 (5h14m54.657302798s) [upload=5396]: Done starting upload Aug 7 21:41:41.070459143 (5h14m54.657515385s) [upload=5396]: Uploading segments... Aug 7 21:41:41.070697238 (5h14m54.657753479s) [upload=5396]: Got next segment. Inline: false Aug 7 21:41:41.070732264 (5h14m54.657788505s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:41.070737293 (5h14m54.657793524s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:41:41.825834701 (5h14m55.412890922s) [seg_pos={0 0}, upload=5396]: Begin upload segment... Aug 7 21:41:41.825865027 (5h14m55.412921237s) [seg_pos={0 0}, upload=5396]: Done begin upload segment. Aug 7 21:41:41.825886015 (5h14m55.412942237s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.825988818 (5h14m55.413045068s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826203979 (5h14m55.413260201s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826435742 (5h14m55.413491953s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826541530 (5h14m55.41359777s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826653037 (5h14m55.413709248s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826875332 (5h14m55.413931543s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826921499 (5h14m55.41397774s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827091606 (5h14m55.414147827s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827323819 (5h14m55.41438003s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827393500 (5h14m55.414449742s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827542938 (5h14m55.414599159s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827778078 (5h14m55.414834318s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827801491 (5h14m55.414857712s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828365735 (5h14m55.415421976s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828913136 (5h14m55.415969378s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.934607129 (5h14m55.52166334s) [upload=5396]: Next returned nil segment Aug 7 21:41:41.934619282 (5h14m55.521675503s) [upload=5396]: Waiting for error group managing segments... Aug 7 21:41:41.945118551 (5h14m55.532174772s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.945901252 (5h14m55.532957473s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946161258 (5h14m55.533217498s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946386648 (5h14m55.533442869s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949024992 (5h14m55.536081243s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949218724 (5h14m55.536274945s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949657033 (5h14m55.536713244s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949683371 (5h14m55.536739612s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.950070354 (5h14m55.537126575s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.953623165 (5h14m55.540679407s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.957708261 (5h14m55.544764482s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964243811 (5h14m55.551300021s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964842018 (5h14m55.551898239s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964860783 (5h14m55.551917004s) [seg_pos={0 0}, upload=5396]: Segment reached optimal threshold of 13 pieces. Aug 7 21:41:41.964962292 (5h14m55.552018503s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965019739 (5h14m55.55207595s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965087696 (5h14m55.552143947s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965118783 (5h14m55.552175005s) [seg_pos={0 0}, upload=5396]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:41:41.965239138 (5h14m55.552295349s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:41:41.965243687 (5h14m55.552299898s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:46.509965896 (5h15m0.097022118s) [upload=5396]: Done uploading segments... Aug 7 21:41:46.510002955 (5h15m0.097059167s) [upload=5396]: Upload finished. err: <nil> Aug 7 21:42:01.196715658 (5h15m14.783771899s) [upload=5401]: Starting upload Aug 7 21:42:01.196842946 (5h15m14.783899187s) [upload=5401]: Done starting upload Aug 7 21:42:01.196915852 (5h15m14.783972103s) [upload=5401]: Uploading segments... Aug 7 21:42:01.197283067 (5h15m14.784339309s) [upload=5401]: Got next segment. Inline: false Aug 7 21:42:01.197355652 (5h15m14.784411984s) [upload=5401]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:01.197365230 (5h15m14.784421472s) [upload=5401]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:42:31.170606605 (5h15m44.757662826s) [upload=5401]: Next segment err: context deadline exceeded Aug 7 21:42:31.170727651 (5h15m44.757783872s) [upload=5401]: Done uploading segments... Aug 7 21:42:31.170749551 (5h15m44.757805773s) [upload=5401]: Upload finished. err: context deadline exceeded Duration:* 557.686729ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 33 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXX..........................] duration: 167.109783ms FAIL (2025-08-07 22:08:48 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== shed. err: metaclient: context deadline exceeded Aug 7 21:38:26.513648977 (5h11m40.100705218s) [upload=5344]: Starting upload Aug 7 21:38:26.513731310 (5h11m40.100787552s) [upload=5344]: Done starting upload Aug 7 21:38:26.513792245 (5h11m40.100848496s) [upload=5344]: Uploading segments... Aug 7 21:38:26.513834112 (5h11m40.100890354s) [upload=5344]: Got next segment. Inline: true Aug 7 21:38:26.513839653 (5h11m40.100895904s) [upload=5344]: Waiting for error group managing segments... Aug 7 21:38:26.513884006 (5h11m40.100940247s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:26.513889947 (5h11m40.100946188s) [upload=5344]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:26.513893434 (5h11m40.100949675s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:56.502052245 (5h12m10.089108537s) [upload=5344]: Done uploading segments... Aug 7 21:38:56.502102730 (5h12m10.089158972s) [upload=5344]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:24.008006172 (5h12m37.595062394s) [upload=5359]: Starting upload Aug 7 21:39:24.008059211 (5h12m37.595115422s) [upload=5359]: Done starting upload Aug 7 21:39:24.008103604 (5h12m37.595159835s) [upload=5359]: Uploading segments... Aug 7 21:39:24.008156191 (5h12m37.595212413s) [upload=5359]: Got next segment. Inline: true Aug 7 21:39:24.008162193 (5h12m37.595218414s) [upload=5359]: Waiting for error group managing segments... Aug 7 21:39:24.008207327 (5h12m37.595263548s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:24.008212668 (5h12m37.595268878s) [upload=5359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:24.008216064 (5h12m37.595272285s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:53.997226503 (5h13m7.584282744s) [upload=5359]: Done uploading segments... Aug 7 21:39:53.997270334 (5h13m7.584326586s) [upload=5359]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:36.779619947 (5h13m50.366676188s) [upload=5383]: Starting upload Aug 7 21:40:36.779696139 (5h13m50.36675238s) [upload=5383]: Done starting upload Aug 7 21:40:36.779752775 (5h13m50.366809016s) [upload=5383]: Uploading segments... Aug 7 21:40:36.779788803 (5h13m50.366845044s) [upload=5383]: Got next segment. Inline: true Aug 7 21:40:36.779794423 (5h13m50.366850655s) [upload=5383]: Waiting for error group managing segments... Aug 7 21:40:36.779835950 (5h13m50.366892191s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:36.779841010 (5h13m50.366897241s) [upload=5383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:36.779844195 (5h13m50.366900437s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:06.765292852 (5h14m20.352349064s) [upload=5383]: Done uploading segments... Aug 7 21:41:06.765314052 (5h14m20.352370273s) [upload=5383]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:47.961322510 (5h15m1.548378802s) [upload=5399]: Starting upload Aug 7 21:41:47.961442915 (5h15m1.548499166s) [upload=5399]: Done starting upload Aug 7 21:41:47.961510201 (5h15m1.548566443s) [upload=5399]: Uploading segments... Aug 7 21:41:47.961591062 (5h15m1.548647313s) [upload=5399]: Got next segment. Inline: true Aug 7 21:41:47.961599218 (5h15m1.548655458s) [upload=5399]: Waiting for error group managing segments... Aug 7 21:41:47.961647759 (5h15m1.548703999s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:47.961653760 (5h15m1.548710001s) [upload=5399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:47.961657546 (5h15m1.548713787s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:17.948677724 (5h15m31.535733945s) [upload=5399]: Done uploading segments... Aug 7 21:42:17.948691099 (5h15m31.53574731s) [upload=5399]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.081006705 (5h15m31.668062915s) [upload=5405]: Starting upload Aug 7 21:42:18.081065655 (5h15m31.668121866s) [upload=5405]: Done starting upload Aug 7 21:42:18.081113425 (5h15m31.668169635s) [upload=5405]: Uploading segments... Aug 7 21:42:18.081164870 (5h15m31.668221091s) [upload=5405]: Got next segment. Inline: true Aug 7 21:42:18.081169459 (5h15m31.66822568s) [upload=5405]: Waiting for error group managing segments... Aug 7 21:42:18.081201809 (5h15m31.66825803s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.081205546 (5h15m31.668261757s) [upload=5405]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.081208271 (5h15m31.668264483s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:18.161269758 (5h15m31.748325999s) [upload=5405]: Done uploading segments... Aug 7 21:42:18.161287411 (5h15m31.748343652s) [upload=5405]: Upload finished. err: <nil> Aug 7 21:43:12.878567983 (5h16m26.465624204s) [upload=5420]: Starting upload Aug 7 21:43:12.878678820 (5h16m26.465735041s) [upload=5420]: Done starting upload Aug 7 21:43:12.878731839 (5h16m26.46578806s) [upload=5420]: Uploading segments... Aug 7 21:43:12.878762908 (5h16m26.465819129s) [upload=5420]: Got next segment. Inline: true Aug 7 21:43:12.878766635 (5h16m26.465822856s) [upload=5420]: Waiting for error group managing segments... Aug 7 21:43:12.878799204 (5h16m26.465855425s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:12.878803332 (5h16m26.465859543s) [upload=5420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:12.878805566 (5h16m26.465861787s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:42.863779226 (5h16m56.450835478s) [upload=5420]: Done uploading segments... Aug 7 21:43:42.863802450 (5h16m56.450858691s) [upload=5420]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:51.464096058 (5h17m5.051152299s) [upload=5431]: Starting upload Aug 7 21:43:51.464210432 (5h17m5.051266673s) [upload=5431]: Done starting upload Aug 7 21:43:51.464274041 (5h17m5.051330282s) [upload=5431]: Uploading segments... Aug 7 21:43:51.464330065 (5h17m5.051386366s) [upload=5431]: Got next segment. Inline: true Aug 7 21:43:51.464343630 (5h17m5.051399912s) [upload=5431]: Waiting for error group managing segments... Aug 7 21:43:51.464415315 (5h17m5.051471556s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:51.464424673 (5h17m5.051480923s) [upload=5431]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:51.464431204 (5h17m5.051487455s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:21.448488824 (5h17m35.035545036s) [upload=5431]: Done uploading segments... Aug 7 21:44:21.448528209 (5h17m35.03558443s) [upload=5431]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:50.396194570 (5h18m3.983250782s) [upload=5444]: Starting upload Aug 7 21:44:50.396243653 (5h18m3.983299864s) [upload=5444]: Done starting upload Aug 7 21:44:50.396286463 (5h18m3.983342684s) [upload=5444]: Uploading segments... Aug 7 21:44:50.396309054 (5h18m3.983365266s) [upload=5444]: Got next segment. Inline: true Aug 7 21:44:50.396312381 (5h18m3.983368592s) [upload=5444]: Waiting for error group managing segments... Aug 7 21:44:50.396342687 (5h18m3.983398908s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:50.396346635 (5h18m3.983402856s) [upload=5444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:50.396348889 (5h18m3.98340511s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:20.384461417 (5h18m33.971517668s) [upload=5444]: Done uploading segments... Aug 7 21:45:20.384482547 (5h18m33.971538788s) [upload=5444]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:58.414522311 (5h19m12.001578522s) [upload=5465]: Starting upload Aug 7 21:45:58.414582292 (5h19m12.001638504s) [upload=5465]: Done starting upload Aug 7 21:45:58.414634661 (5h19m12.001690882s) [upload=5465]: Uploading segments... Aug 7 21:45:58.414662251 (5h19m12.001718462s) [upload=5465]: Got next segment. Inline: true Aug 7 21:45:58.414665918 (5h19m12.00172213s) [upload=5465]: Waiting for error group managing segments... Aug 7 21:45:58.414697247 (5h19m12.001753468s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.414700753 (5h19m12.001756975s) [upload=5465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:58.414703007 (5h19m12.001759229s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:28.397843251 (5h19m41.984899492s) [upload=5465]: Done uploading segments... Aug 7 21:46:28.397885870 (5h19m41.984942112s) [upload=5465]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:07.964355881 (5h20m21.551412133s) [upload=5484]: Starting upload Aug 7 21:47:07.964442332 (5h20m21.551498584s) [upload=5484]: Done starting upload Aug 7 21:47:07.964529184 (5h20m21.551585426s) [upload=5484]: Uploading segments... Aug 7 21:47:07.964568658 (5h20m21.5516249s) [upload=5484]: Got next segment. Inline: true Aug 7 21:47:07.964573968 (5h20m21.551630209s) [upload=5484]: Waiting for error group managing segments... Aug 7 21:47:07.964617630 (5h20m21.551673881s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:07.964623640 (5h20m21.551679882s) [upload=5484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:07.964628048 (5h20m21.55168429s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:37.944795042 (5h20m51.531851283s) [upload=5484]: Done uploading segments... Aug 7 21:47:37.944820539 (5h20m51.531876781s) [upload=5484]: Upload finished. err: metaclient: context deadline exceeded Duration: 167.126675ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXX.................................] duration: 1.697933216s FAIL (2025-08-07 22:08:48 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.697956039s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [....XX...X..................................................] duration: 20.653403ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:08:35 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 20.665165ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [...........X...X...X........................................] duration: 130.310543ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:08:30 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 130.329538ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 71.199184ms FAIL (2025-08-07 22:08:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ine Aug 7 21:59:56.515336538 (5h33m10.10239275s) [upload=5708]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:59:56.574152557 (5h33m10.161208778s) [upload=5708]: Done uploading segments... Aug 7 21:59:56.574171943 (5h33m10.161228164s) [upload=5708]: Upload finished. err: <nil> Aug 7 22:00:36.611785100 (5h33m50.19884134s) [upload=5716]: Starting upload Aug 7 22:00:36.611856203 (5h33m50.198912444s) [upload=5716]: Done starting upload Aug 7 22:00:36.611959786 (5h33m50.199016027s) [upload=5716]: Uploading segments... Aug 7 22:00:36.612003668 (5h33m50.199059918s) [upload=5716]: Got next segment. Inline: true Aug 7 22:00:36.612009668 (5h33m50.19906591s) [upload=5716]: Waiting for error group managing segments... Aug 7 22:00:36.612057077 (5h33m50.199113319s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:00:36.612082536 (5h33m50.199138776s) [upload=5716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:00:36.612086222 (5h33m50.199142473s) [upload=5716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:00:36.884781084 (5h33m50.471837294s) [upload=5716]: Done uploading segments... Aug 7 22:00:36.884795611 (5h33m50.471851832s) [upload=5716]: Upload finished. err: <nil> Aug 7 22:01:47.555449092 (5h35m1.142505313s) [upload=5736]: Starting upload Aug 7 22:01:47.555491781 (5h35m1.142548002s) [upload=5736]: Done starting upload Aug 7 22:01:47.555562123 (5h35m1.142618344s) [upload=5736]: Uploading segments... Aug 7 22:01:47.555595826 (5h35m1.142652047s) [upload=5736]: Got next segment. Inline: true Aug 7 22:01:47.555599363 (5h35m1.142655584s) [upload=5736]: Waiting for error group managing segments... Aug 7 22:01:47.555631602 (5h35m1.142687823s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:01:47.555635390 (5h35m1.142691611s) [upload=5736]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:01:47.555637894 (5h35m1.142694115s) [upload=5736]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:01:47.686110188 (5h35m1.273166429s) [upload=5736]: Done uploading segments... Aug 7 22:01:47.686151165 (5h35m1.273207406s) [upload=5736]: Upload finished. err: <nil> Aug 7 22:02:22.678566055 (5h35m36.265622267s) [upload=5746]: Starting upload Aug 7 22:02:22.678633091 (5h35m36.265689303s) [upload=5746]: Done starting upload Aug 7 22:02:22.678684407 (5h35m36.265740617s) [upload=5746]: Uploading segments... Aug 7 22:02:22.678721757 (5h35m36.265777967s) [upload=5746]: Got next segment. Inline: true Aug 7 22:02:22.678725724 (5h35m36.265781935s) [upload=5746]: Waiting for error group managing segments... Aug 7 22:02:22.678759457 (5h35m36.265815668s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:02:22.678764947 (5h35m36.265821168s) [upload=5746]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:02:22.678768565 (5h35m36.265824826s) [upload=5746]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:02:22.971349697 (5h35m36.558405918s) [upload=5746]: Done uploading segments... Aug 7 22:02:22.971373742 (5h35m36.558429963s) [upload=5746]: Upload finished. err: <nil> Aug 7 22:03:08.907032448 (5h36m22.494088689s) [upload=5757]: Starting upload Aug 7 22:03:08.907091528 (5h36m22.494147769s) [upload=5757]: Done starting upload Aug 7 22:03:08.907172368 (5h36m22.4942286s) [upload=5757]: Uploading segments... Aug 7 22:03:08.907209458 (5h36m22.494265699s) [upload=5757]: Got next segment. Inline: true Aug 7 22:03:08.907214758 (5h36m22.494271s) [upload=5757]: Waiting for error group managing segments... Aug 7 22:03:08.907255936 (5h36m22.494312176s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:03:08.907261175 (5h36m22.494317416s) [upload=5757]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:03:08.907265904 (5h36m22.494322135s) [upload=5757]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:03:09.004207779 (5h36m22.59126404s) [upload=5757]: Done uploading segments... Aug 7 22:03:09.004232375 (5h36m22.591288627s) [upload=5757]: Upload finished. err: <nil> Aug 7 22:04:02.463086309 (5h37m16.050142521s) [upload=5775]: Starting upload Aug 7 22:04:02.463128347 (5h37m16.050184568s) [upload=5775]: Done starting upload Aug 7 22:04:02.463197818 (5h37m16.050254029s) [upload=5775]: Uploading segments... Aug 7 22:04:02.463223595 (5h37m16.050279816s) [upload=5775]: Got next segment. Inline: true Aug 7 22:04:02.463227652 (5h37m16.050283873s) [upload=5775]: Waiting for error group managing segments... Aug 7 22:04:02.463258120 (5h37m16.050314331s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:04:02.463261796 (5h37m16.050318008s) [upload=5775]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:04:02.463264733 (5h37m16.050320954s) [upload=5775]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:04:02.530547234 (5h37m16.117603445s) [upload=5775]: Done uploading segments... Aug 7 22:04:02.530562512 (5h37m16.117618733s) [upload=5775]: Upload finished. err: <nil> Aug 7 22:05:10.744065032 (5h38m24.331121272s) [upload=5796]: Starting upload Aug 7 22:05:10.744195716 (5h38m24.331251957s) [upload=5796]: Done starting upload Aug 7 22:05:10.744327762 (5h38m24.331384002s) [upload=5796]: Uploading segments... Aug 7 22:05:10.744369549 (5h38m24.331425791s) [upload=5796]: Got next segment. Inline: true Aug 7 22:05:10.744388655 (5h38m24.331444897s) [upload=5796]: Waiting for error group managing segments... Aug 7 22:05:10.744449028 (5h38m24.331505259s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:10.744454398 (5h38m24.331510629s) [upload=5796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:10.744457633 (5h38m24.331513875s) [upload=5796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:10.852991578 (5h38m24.440047789s) [upload=5796]: Done uploading segments... Aug 7 22:05:10.853004571 (5h38m24.440060782s) [upload=5796]: Upload finished. err: <nil> Aug 7 22:05:45.060720865 (5h38m58.647777086s) [upload=5806]: Starting upload Aug 7 22:05:45.060780046 (5h38m58.647836267s) [upload=5806]: Done starting upload Aug 7 22:05:45.060842121 (5h38m58.647898333s) [upload=5806]: Uploading segments... Aug 7 22:05:45.060876836 (5h38m58.647933057s) [upload=5806]: Got next segment. Inline: true Aug 7 22:05:45.060882878 (5h38m58.647939099s) [upload=5806]: Waiting for error group managing segments... Aug 7 22:05:45.060927901 (5h38m58.647984112s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:05:45.060933843 (5h38m58.647990064s) [upload=5806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:05:45.060936187 (5h38m58.647992408s) [upload=5806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:05:45.328399453 (5h38m58.915455674s) [upload=5806]: Done uploading segments... Aug 7 22:05:45.328433367 (5h38m58.915489588s) [upload=5806]: Upload finished. err: <nil> Aug 7 22:06:12.931860875 (5h39m26.518917096s) [upload=5815]: Starting upload Aug 7 22:06:12.931905979 (5h39m26.51896219s) [upload=5815]: Done starting upload Aug 7 22:06:12.931964518 (5h39m26.51902073s) [upload=5815]: Uploading segments... Aug 7 22:06:12.931990617 (5h39m26.519046827s) [upload=5815]: Got next segment. Inline: true Aug 7 22:06:12.931994224 (5h39m26.519050434s) [upload=5815]: Waiting for error group managing segments... Aug 7 22:06:12.932024821 (5h39m26.519081042s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:06:12.932028408 (5h39m26.519084618s) [upload=5815]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:06:12.932030893 (5h39m26.519087103s) [upload=5815]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:06:13.014995076 (5h39m26.602051317s) [upload=5815]: Done uploading segments... Aug 7 22:06:13.015014863 (5h39m26.602071114s) [upload=5815]: Upload finished. err: <nil> Aug 7 22:07:16.916734820 (5h40m30.503791062s) [upload=5832]: Starting upload Aug 7 22:07:16.916800323 (5h40m30.503856563s) [upload=5832]: Done starting upload Aug 7 22:07:16.916864883 (5h40m30.503921123s) [upload=5832]: Uploading segments... Aug 7 22:07:16.916903304 (5h40m30.503959546s) [upload=5832]: Got next segment. Inline: true Aug 7 22:07:16.916908815 (5h40m30.503965056s) [upload=5832]: Waiting for error group managing segments... Aug 7 22:07:16.916952356 (5h40m30.504008598s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:07:16.916957657 (5h40m30.504013897s) [upload=5832]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:07:16.916961754 (5h40m30.504017995s) [upload=5832]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:07:16.988776316 (5h40m30.575832566s) [upload=5832]: Done uploading segments... Aug 7 22:07:16.988801573 (5h40m30.575857824s) [upload=5832]: Upload finished. err: <nil> Aug 7 22:08:29.650294124 (5h41m43.237350365s) [upload=5854]: Starting upload Aug 7 22:08:29.650357383 (5h41m43.237413623s) [upload=5854]: Done starting upload Aug 7 22:08:29.650418325 (5h41m43.237474567s) [upload=5854]: Uploading segments... Aug 7 22:08:29.650468490 (5h41m43.237524731s) [upload=5854]: Got next segment. Inline: true Aug 7 22:08:29.650474350 (5h41m43.237530592s) [upload=5854]: Waiting for error group managing segments... Aug 7 22:08:29.650515618 (5h41m43.237571859s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 22:08:29.650522029 (5h41m43.237578271s) [upload=5854]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 22:08:29.650525556 (5h41m43.237581797s) [upload=5854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 22:08:29.706715553 (5h41m43.293771793s) [upload=5854]: Done uploading segments... Aug 7 22:08:29.706731001 (5h41m43.293787243s) [upload=5854]: Upload finished. err: metaclient: Too Many Requests Duration:* 71.234409ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 20 out of 60 is failed (threshold=3) [......XXXXXXXXXXXXXXXXX.XX......X...........................] duration: 451.03506ms FAIL (2025-08-07 22:08:27 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== .251202997s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:13.664150523 (5h20m27.251206734s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:57.268638109 (5h21m10.855694349s) [seg_pos={0 0}, upload=5487]: Begin upload segment... Aug 7 21:47:57.268679396 (5h21m10.855735637s) [seg_pos={0 0}, upload=5487]: Done begin upload segment. Aug 7 21:47:57.268708491 (5h21m10.855764731s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.268768562 (5h21m10.855824803s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269205608 (5h21m10.856261849s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269223562 (5h21m10.856279802s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269609953 (5h21m10.856666204s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269616095 (5h21m10.856672345s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269967370 (5h21m10.857023612s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269989231 (5h21m10.857045472s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270389678 (5h21m10.85744593s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270397674 (5h21m10.857453915s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270761794 (5h21m10.857818034s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270773264 (5h21m10.857829505s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271108780 (5h21m10.858165022s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271123869 (5h21m10.85818011s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271518095 (5h21m10.858574346s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272004884 (5h21m10.859061125s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272493695 (5h21m10.859549937s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.273356537 (5h21m10.860412828s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.396251337 (5h21m10.983307579s) [upload=5487]: Next returned nil segment Aug 7 21:47:57.396270474 (5h21m10.983326714s) [upload=5487]: Waiting for error group managing segments... Aug 7 21:47:57.416691513 (5h21m11.003747763s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.420507917 (5h21m11.007564157s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.425516195 (5h21m11.012572446s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.427782495 (5h21m11.014838736s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.431475158 (5h21m11.018531398s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.432679355 (5h21m11.019735596s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.449481709 (5h21m11.036537951s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.451301637 (5h21m11.038357878s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.469893221 (5h21m11.056949472s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.470298257 (5h21m11.057354497s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.475204656 (5h21m11.062260917s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.549461095 (5h21m11.136517316s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.661813247 (5h21m11.248869498s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755309651 (5h21m11.342365892s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755326954 (5h21m11.342383194s) [seg_pos={0 0}, upload=5487]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:57.755590967 (5h21m11.342647207s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755687266 (5h21m11.342743507s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUV64TP8cTQhBjswnhcWfQHwYEEWqr5ASuPjtEx3X6ZQNVhS5): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755708706 (5h21m11.342764947s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection Aug 7 21:47:57.755820425 (5h21m11.342876656s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WKWWwa3svATpoNY1z7LNvYmA8gA6ZFpJsk17G4c37tDJVJ2wj): protocol: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection Aug 7 21:47:57.755837257 (5h21m11.342893498s) [seg_pos={0 0}, upload=5487]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:57.755901476 (5h21m11.342957717s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:57.755906926 (5h21m11.342963168s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:45.449359965 (5h21m59.036416176s) [upload=5487]: Done uploading segments... Aug 7 21:48:45.449390482 (5h21m59.036446703s) [upload=5487]: Upload finished. err: metaclient: context deadline exceeded Duration: 451.05149ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 25 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXX...X.....X...........................] duration: 246.301035ms FAIL (2025-08-07 22:08:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 246.318748ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 28 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX................................] duration: 1.869560041s FAIL (2025-08-07 22:08:15 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.8695966s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.....XX...X.................................................] duration: 39.8956ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:08:15 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 39.919114ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [.XXXXXXXXXXXXXXXXXXXXXXXXXX.................................] duration: 1.639763445s FAIL (2025-08-07 22:08:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.63979839s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXX.XX.....................................] duration: 529.903046ms FAIL (2025-08-07 22:08:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 529.978125ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 34 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXX.........................] duration: 196.114012ms FAIL (2025-08-07 22:08:08 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== shed. err: metaclient: context deadline exceeded Aug 7 21:38:26.513648977 (5h11m40.100705218s) [upload=5344]: Starting upload Aug 7 21:38:26.513731310 (5h11m40.100787552s) [upload=5344]: Done starting upload Aug 7 21:38:26.513792245 (5h11m40.100848496s) [upload=5344]: Uploading segments... Aug 7 21:38:26.513834112 (5h11m40.100890354s) [upload=5344]: Got next segment. Inline: true Aug 7 21:38:26.513839653 (5h11m40.100895904s) [upload=5344]: Waiting for error group managing segments... Aug 7 21:38:26.513884006 (5h11m40.100940247s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:26.513889947 (5h11m40.100946188s) [upload=5344]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:26.513893434 (5h11m40.100949675s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:56.502052245 (5h12m10.089108537s) [upload=5344]: Done uploading segments... Aug 7 21:38:56.502102730 (5h12m10.089158972s) [upload=5344]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:24.008006172 (5h12m37.595062394s) [upload=5359]: Starting upload Aug 7 21:39:24.008059211 (5h12m37.595115422s) [upload=5359]: Done starting upload Aug 7 21:39:24.008103604 (5h12m37.595159835s) [upload=5359]: Uploading segments... Aug 7 21:39:24.008156191 (5h12m37.595212413s) [upload=5359]: Got next segment. Inline: true Aug 7 21:39:24.008162193 (5h12m37.595218414s) [upload=5359]: Waiting for error group managing segments... Aug 7 21:39:24.008207327 (5h12m37.595263548s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:24.008212668 (5h12m37.595268878s) [upload=5359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:24.008216064 (5h12m37.595272285s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:53.997226503 (5h13m7.584282744s) [upload=5359]: Done uploading segments... Aug 7 21:39:53.997270334 (5h13m7.584326586s) [upload=5359]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:36.779619947 (5h13m50.366676188s) [upload=5383]: Starting upload Aug 7 21:40:36.779696139 (5h13m50.36675238s) [upload=5383]: Done starting upload Aug 7 21:40:36.779752775 (5h13m50.366809016s) [upload=5383]: Uploading segments... Aug 7 21:40:36.779788803 (5h13m50.366845044s) [upload=5383]: Got next segment. Inline: true Aug 7 21:40:36.779794423 (5h13m50.366850655s) [upload=5383]: Waiting for error group managing segments... Aug 7 21:40:36.779835950 (5h13m50.366892191s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:36.779841010 (5h13m50.366897241s) [upload=5383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:36.779844195 (5h13m50.366900437s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:06.765292852 (5h14m20.352349064s) [upload=5383]: Done uploading segments... Aug 7 21:41:06.765314052 (5h14m20.352370273s) [upload=5383]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:47.961322510 (5h15m1.548378802s) [upload=5399]: Starting upload Aug 7 21:41:47.961442915 (5h15m1.548499166s) [upload=5399]: Done starting upload Aug 7 21:41:47.961510201 (5h15m1.548566443s) [upload=5399]: Uploading segments... Aug 7 21:41:47.961591062 (5h15m1.548647313s) [upload=5399]: Got next segment. Inline: true Aug 7 21:41:47.961599218 (5h15m1.548655458s) [upload=5399]: Waiting for error group managing segments... Aug 7 21:41:47.961647759 (5h15m1.548703999s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:47.961653760 (5h15m1.548710001s) [upload=5399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:47.961657546 (5h15m1.548713787s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:17.948677724 (5h15m31.535733945s) [upload=5399]: Done uploading segments... Aug 7 21:42:17.948691099 (5h15m31.53574731s) [upload=5399]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.081006705 (5h15m31.668062915s) [upload=5405]: Starting upload Aug 7 21:42:18.081065655 (5h15m31.668121866s) [upload=5405]: Done starting upload Aug 7 21:42:18.081113425 (5h15m31.668169635s) [upload=5405]: Uploading segments... Aug 7 21:42:18.081164870 (5h15m31.668221091s) [upload=5405]: Got next segment. Inline: true Aug 7 21:42:18.081169459 (5h15m31.66822568s) [upload=5405]: Waiting for error group managing segments... Aug 7 21:42:18.081201809 (5h15m31.66825803s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.081205546 (5h15m31.668261757s) [upload=5405]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.081208271 (5h15m31.668264483s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:18.161269758 (5h15m31.748325999s) [upload=5405]: Done uploading segments... Aug 7 21:42:18.161287411 (5h15m31.748343652s) [upload=5405]: Upload finished. err: <nil> Aug 7 21:43:12.878567983 (5h16m26.465624204s) [upload=5420]: Starting upload Aug 7 21:43:12.878678820 (5h16m26.465735041s) [upload=5420]: Done starting upload Aug 7 21:43:12.878731839 (5h16m26.46578806s) [upload=5420]: Uploading segments... Aug 7 21:43:12.878762908 (5h16m26.465819129s) [upload=5420]: Got next segment. Inline: true Aug 7 21:43:12.878766635 (5h16m26.465822856s) [upload=5420]: Waiting for error group managing segments... Aug 7 21:43:12.878799204 (5h16m26.465855425s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:12.878803332 (5h16m26.465859543s) [upload=5420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:12.878805566 (5h16m26.465861787s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:42.863779226 (5h16m56.450835478s) [upload=5420]: Done uploading segments... Aug 7 21:43:42.863802450 (5h16m56.450858691s) [upload=5420]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:51.464096058 (5h17m5.051152299s) [upload=5431]: Starting upload Aug 7 21:43:51.464210432 (5h17m5.051266673s) [upload=5431]: Done starting upload Aug 7 21:43:51.464274041 (5h17m5.051330282s) [upload=5431]: Uploading segments... Aug 7 21:43:51.464330065 (5h17m5.051386366s) [upload=5431]: Got next segment. Inline: true Aug 7 21:43:51.464343630 (5h17m5.051399912s) [upload=5431]: Waiting for error group managing segments... Aug 7 21:43:51.464415315 (5h17m5.051471556s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:51.464424673 (5h17m5.051480923s) [upload=5431]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:51.464431204 (5h17m5.051487455s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:21.448488824 (5h17m35.035545036s) [upload=5431]: Done uploading segments... Aug 7 21:44:21.448528209 (5h17m35.03558443s) [upload=5431]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:50.396194570 (5h18m3.983250782s) [upload=5444]: Starting upload Aug 7 21:44:50.396243653 (5h18m3.983299864s) [upload=5444]: Done starting upload Aug 7 21:44:50.396286463 (5h18m3.983342684s) [upload=5444]: Uploading segments... Aug 7 21:44:50.396309054 (5h18m3.983365266s) [upload=5444]: Got next segment. Inline: true Aug 7 21:44:50.396312381 (5h18m3.983368592s) [upload=5444]: Waiting for error group managing segments... Aug 7 21:44:50.396342687 (5h18m3.983398908s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:50.396346635 (5h18m3.983402856s) [upload=5444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:50.396348889 (5h18m3.98340511s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:20.384461417 (5h18m33.971517668s) [upload=5444]: Done uploading segments... Aug 7 21:45:20.384482547 (5h18m33.971538788s) [upload=5444]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:58.414522311 (5h19m12.001578522s) [upload=5465]: Starting upload Aug 7 21:45:58.414582292 (5h19m12.001638504s) [upload=5465]: Done starting upload Aug 7 21:45:58.414634661 (5h19m12.001690882s) [upload=5465]: Uploading segments... Aug 7 21:45:58.414662251 (5h19m12.001718462s) [upload=5465]: Got next segment. Inline: true Aug 7 21:45:58.414665918 (5h19m12.00172213s) [upload=5465]: Waiting for error group managing segments... Aug 7 21:45:58.414697247 (5h19m12.001753468s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.414700753 (5h19m12.001756975s) [upload=5465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:58.414703007 (5h19m12.001759229s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:28.397843251 (5h19m41.984899492s) [upload=5465]: Done uploading segments... Aug 7 21:46:28.397885870 (5h19m41.984942112s) [upload=5465]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:07.964355881 (5h20m21.551412133s) [upload=5484]: Starting upload Aug 7 21:47:07.964442332 (5h20m21.551498584s) [upload=5484]: Done starting upload Aug 7 21:47:07.964529184 (5h20m21.551585426s) [upload=5484]: Uploading segments... Aug 7 21:47:07.964568658 (5h20m21.5516249s) [upload=5484]: Got next segment. Inline: true Aug 7 21:47:07.964573968 (5h20m21.551630209s) [upload=5484]: Waiting for error group managing segments... Aug 7 21:47:07.964617630 (5h20m21.551673881s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:07.964623640 (5h20m21.551679882s) [upload=5484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:07.964628048 (5h20m21.55168429s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:37.944795042 (5h20m51.531851283s) [upload=5484]: Done uploading segments... Aug 7 21:47:37.944820539 (5h20m51.531876781s) [upload=5484]: Upload finished. err: metaclient: context deadline exceeded Duration: 196.140112ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 26 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXX...X.....X..........................] duration: 202.08468ms FAIL (2025-08-07 22:08:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 202.115728ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXX..X...................................] duration: 478.32882ms FAIL (2025-08-07 22:08:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 (5h14m28.062269058s) [upload=5388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:25.266685321 (5h14m38.853741562s) [upload=5388]: Done uploading segments... Aug 7 21:41:25.266706391 (5h14m38.853762631s) [upload=5388]: Upload finished. err: <nil> Aug 7 21:41:41.070176076 (5h14m54.657232316s) [upload=5396]: Starting upload Aug 7 21:41:41.070246557 (5h14m54.657302798s) [upload=5396]: Done starting upload Aug 7 21:41:41.070459143 (5h14m54.657515385s) [upload=5396]: Uploading segments... Aug 7 21:41:41.070697238 (5h14m54.657753479s) [upload=5396]: Got next segment. Inline: false Aug 7 21:41:41.070732264 (5h14m54.657788505s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:41.070737293 (5h14m54.657793524s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:41:41.825834701 (5h14m55.412890922s) [seg_pos={0 0}, upload=5396]: Begin upload segment... Aug 7 21:41:41.825865027 (5h14m55.412921237s) [seg_pos={0 0}, upload=5396]: Done begin upload segment. Aug 7 21:41:41.825886015 (5h14m55.412942237s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.825988818 (5h14m55.413045068s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826203979 (5h14m55.413260201s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826435742 (5h14m55.413491953s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826541530 (5h14m55.41359777s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826653037 (5h14m55.413709248s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826875332 (5h14m55.413931543s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826921499 (5h14m55.41397774s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827091606 (5h14m55.414147827s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827323819 (5h14m55.41438003s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827393500 (5h14m55.414449742s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827542938 (5h14m55.414599159s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827778078 (5h14m55.414834318s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827801491 (5h14m55.414857712s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828365735 (5h14m55.415421976s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828913136 (5h14m55.415969378s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.934607129 (5h14m55.52166334s) [upload=5396]: Next returned nil segment Aug 7 21:41:41.934619282 (5h14m55.521675503s) [upload=5396]: Waiting for error group managing segments... Aug 7 21:41:41.945118551 (5h14m55.532174772s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.945901252 (5h14m55.532957473s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946161258 (5h14m55.533217498s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946386648 (5h14m55.533442869s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949024992 (5h14m55.536081243s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949218724 (5h14m55.536274945s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949657033 (5h14m55.536713244s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949683371 (5h14m55.536739612s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.950070354 (5h14m55.537126575s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.953623165 (5h14m55.540679407s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.957708261 (5h14m55.544764482s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964243811 (5h14m55.551300021s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964842018 (5h14m55.551898239s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964860783 (5h14m55.551917004s) [seg_pos={0 0}, upload=5396]: Segment reached optimal threshold of 13 pieces. Aug 7 21:41:41.964962292 (5h14m55.552018503s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965019739 (5h14m55.55207595s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965087696 (5h14m55.552143947s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965118783 (5h14m55.552175005s) [seg_pos={0 0}, upload=5396]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:41:41.965239138 (5h14m55.552295349s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:41:41.965243687 (5h14m55.552299898s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:46.509965896 (5h15m0.097022118s) [upload=5396]: Done uploading segments... Aug 7 21:41:46.510002955 (5h15m0.097059167s) [upload=5396]: Upload finished. err: <nil> Aug 7 21:42:01.196715658 (5h15m14.783771899s) [upload=5401]: Starting upload Aug 7 21:42:01.196842946 (5h15m14.783899187s) [upload=5401]: Done starting upload Aug 7 21:42:01.196915852 (5h15m14.783972103s) [upload=5401]: Uploading segments... Aug 7 21:42:01.197283067 (5h15m14.784339309s) [upload=5401]: Got next segment. Inline: false Aug 7 21:42:01.197355652 (5h15m14.784411984s) [upload=5401]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:01.197365230 (5h15m14.784421472s) [upload=5401]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:42:31.170606605 (5h15m44.757662826s) [upload=5401]: Next segment err: context deadline exceeded Aug 7 21:42:31.170727651 (5h15m44.757783872s) [upload=5401]: Done uploading segments... Aug 7 21:42:31.170749551 (5h15m44.757805773s) [upload=5401]: Upload finished. err: context deadline exceeded Duration:* 478.364167ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 31 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.XX...........................] duration: 191.1725ms FAIL (2025-08-07 22:07:58 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 31725988s) [upload=5324]: Upload finished. err: <nil> Aug 7 21:38:15.287715124 (5h11m28.874771365s) [upload=5341]: Starting upload Aug 7 21:38:15.287780276 (5h11m28.874836516s) [upload=5341]: Done starting upload Aug 7 21:38:15.287844856 (5h11m28.874901107s) [upload=5341]: Uploading segments... Aug 7 21:38:15.287889961 (5h11m28.874946201s) [upload=5341]: Got next segment. Inline: true Aug 7 21:38:15.287895600 (5h11m28.874951842s) [upload=5341]: Waiting for error group managing segments... Aug 7 21:38:15.287984066 (5h11m28.875040317s) [upload=5341]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:15.287990077 (5h11m28.875046318s) [upload=5341]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:15.287993564 (5h11m28.875049804s) [upload=5341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:15.275679218 (5h12m28.862735459s) [upload=5341]: Done uploading segments... Aug 7 21:39:15.275708622 (5h12m28.862764883s) [upload=5341]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:15.358605080 (5h12m28.945661322s) [upload=5358]: Starting upload Aug 7 21:39:15.358690641 (5h12m28.945746882s) [upload=5358]: Done starting upload Aug 7 21:39:15.358748869 (5h12m28.945805111s) [upload=5358]: Uploading segments... Aug 7 21:39:15.358790938 (5h12m28.94584718s) [upload=5358]: Got next segment. Inline: true Aug 7 21:39:15.358796779 (5h12m28.94585302s) [upload=5358]: Waiting for error group managing segments... Aug 7 21:39:15.358840480 (5h12m28.945896721s) [upload=5358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:15.358846441 (5h12m28.945902683s) [upload=5358]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:15.358849918 (5h12m28.945906159s) [upload=5358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:15.344303433 (5h13m28.931359654s) [upload=5358]: Done uploading segments... Aug 7 21:40:15.344323120 (5h13m28.931379331s) [upload=5358]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:15.438334176 (5h13m29.025390397s) [upload=5373]: Starting upload Aug 7 21:40:15.438379379 (5h13m29.025435601s) [upload=5373]: Done starting upload Aug 7 21:40:15.438418503 (5h13m29.025474714s) [upload=5373]: Uploading segments... Aug 7 21:40:15.438447206 (5h13m29.025503427s) [upload=5373]: Got next segment. Inline: true Aug 7 21:40:15.438450683 (5h13m29.025506894s) [upload=5373]: Waiting for error group managing segments... Aug 7 21:40:15.438481920 (5h13m29.025538141s) [upload=5373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:15.438487511 (5h13m29.025543722s) [upload=5373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:15.438489826 (5h13m29.025546047s) [upload=5373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:15.423387101 (5h14m29.010443342s) [upload=5373]: Done uploading segments... Aug 7 21:41:15.423408011 (5h14m29.010464251s) [upload=5373]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:15.514709386 (5h14m29.101765628s) [upload=5390]: Starting upload Aug 7 21:41:15.514790870 (5h14m29.10184711s) [upload=5390]: Done starting upload Aug 7 21:41:15.514852845 (5h14m29.101909086s) [upload=5390]: Uploading segments... Aug 7 21:41:15.514892409 (5h14m29.10194865s) [upload=5390]: Got next segment. Inline: true Aug 7 21:41:15.514898240 (5h14m29.101954481s) [upload=5390]: Waiting for error group managing segments... Aug 7 21:41:15.514941951 (5h14m29.101998192s) [upload=5390]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:15.514947551 (5h14m29.102003793s) [upload=5390]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:15.514951138 (5h14m29.102007389s) [upload=5390]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:15.497042181 (5h15m29.084098402s) [upload=5390]: Done uploading segments... Aug 7 21:42:15.497058301 (5h15m29.084114512s) [upload=5390]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:15.596278951 (5h15m29.183335172s) [upload=5404]: Starting upload Aug 7 21:42:15.596337329 (5h15m29.183393541s) [upload=5404]: Done starting upload Aug 7 21:42:15.596382114 (5h15m29.183438324s) [upload=5404]: Uploading segments... Aug 7 21:42:15.596411919 (5h15m29.18346814s) [upload=5404]: Got next segment. Inline: true Aug 7 21:42:15.596415937 (5h15m29.183472158s) [upload=5404]: Waiting for error group managing segments... Aug 7 21:42:15.596447405 (5h15m29.183503626s) [upload=5404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:15.596451042 (5h15m29.183507263s) [upload=5404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:15.596453266 (5h15m29.183509487s) [upload=5404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:15.584735444 (5h16m29.171791655s) [upload=5404]: Done uploading segments... Aug 7 21:43:15.584747787 (5h16m29.171804008s) [upload=5404]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:22.691047415 (5h16m36.278103646s) [upload=5423]: Starting upload Aug 7 21:43:22.691110724 (5h16m36.278166955s) [upload=5423]: Done starting upload Aug 7 21:43:22.691177108 (5h16m36.278233338s) [upload=5423]: Uploading segments... Aug 7 21:43:22.691210019 (5h16m36.27826625s) [upload=5423]: Got next segment. Inline: true Aug 7 21:43:22.691214187 (5h16m36.278270418s) [upload=5423]: Waiting for error group managing segments... Aug 7 21:43:22.691250704 (5h16m36.278306925s) [upload=5423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:22.691255263 (5h16m36.278311494s) [upload=5423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:22.691258188 (5h16m36.278314419s) [upload=5423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:22.751511620 (5h16m36.338567872s) [upload=5423]: Done uploading segments... Aug 7 21:43:22.751530204 (5h16m36.338586455s) [upload=5423]: Upload finished. err: <nil> Aug 7 21:44:13.185700222 (5h17m26.772756433s) [upload=5437]: Starting upload Aug 7 21:44:13.185766515 (5h17m26.772822736s) [upload=5437]: Done starting upload Aug 7 21:44:13.185811771 (5h17m26.772867981s) [upload=5437]: Uploading segments... Aug 7 21:44:13.185877192 (5h17m26.772933443s) [upload=5437]: Got next segment. Inline: true Aug 7 21:44:13.185884005 (5h17m26.772940226s) [upload=5437]: Waiting for error group managing segments... Aug 7 21:44:13.185926104 (5h17m26.772982314s) [upload=5437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:13.185931043 (5h17m26.772987254s) [upload=5437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:13.185934560 (5h17m26.77299077s) [upload=5437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:13.172887995 (5h18m26.759944236s) [upload=5437]: Done uploading segments... Aug 7 21:45:13.172908133 (5h18m26.759964374s) [upload=5437]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:13.256441576 (5h18m26.843497827s) [upload=5449]: Starting upload Aug 7 21:45:13.256514653 (5h18m26.843570894s) [upload=5449]: Done starting upload Aug 7 21:45:13.256603699 (5h18m26.84365995s) [upload=5449]: Uploading segments... Aug 7 21:45:13.256644835 (5h18m26.843701076s) [upload=5449]: Got next segment. Inline: true Aug 7 21:45:13.256650195 (5h18m26.843706436s) [upload=5449]: Waiting for error group managing segments... Aug 7 21:45:13.256695270 (5h18m26.843751511s) [upload=5449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:13.256701020 (5h18m26.843757262s) [upload=5449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:13.256704657 (5h18m26.843760898s) [upload=5449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:13.340684827 (5h18m26.927741079s) [upload=5449]: Done uploading segments... Aug 7 21:45:13.340709023 (5h18m26.927765264s) [upload=5449]: Upload finished. err: <nil> Aug 7 21:46:18.644805531 (5h19m32.231861771s) [upload=5472]: Starting upload Aug 7 21:46:18.644902892 (5h19m32.231959133s) [upload=5472]: Done starting upload Aug 7 21:46:18.644968324 (5h19m32.232024565s) [upload=5472]: Uploading segments... Aug 7 21:46:18.645015051 (5h19m32.232071292s) [upload=5472]: Got next segment. Inline: true Aug 7 21:46:18.645020682 (5h19m32.232076933s) [upload=5472]: Waiting for error group managing segments... Aug 7 21:46:18.645064344 (5h19m32.232120585s) [upload=5472]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:18.645070025 (5h19m32.232126266s) [upload=5472]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:18.645073771 (5h19m32.232130013s) [upload=5472]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:18.630338668 (5h20m32.217394879s) [upload=5472]: Done uploading segments... Aug 7 21:47:18.630353927 (5h20m32.217410137s) [upload=5472]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:18.710984265 (5h20m32.298040506s) [upload=5489]: Starting upload Aug 7 21:47:18.711056571 (5h20m32.298112812s) [upload=5489]: Done starting upload Aug 7 21:47:18.711125208 (5h20m32.29818145s) [upload=5489]: Uploading segments... Aug 7 21:47:18.711173729 (5h20m32.29822996s) [upload=5489]: Got next segment. Inline: true Aug 7 21:47:18.711179229 (5h20m32.298235471s) [upload=5489]: Waiting for error group managing segments... Aug 7 21:47:18.711220847 (5h20m32.298277087s) [upload=5489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:18.711225816 (5h20m32.298282057s) [upload=5489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:18.711229112 (5h20m32.298285354s) [upload=5489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:18.698417946 (5h21m32.285474187s) [upload=5489]: Done uploading segments... Aug 7 21:48:18.698439776 (5h21m32.285496017s) [upload=5489]: Upload finished. err: metaclient: context deadline exceeded Duration: 191.207355ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [............X...X...X.......................................] duration: 118.023118ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:07:52 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 118.043415ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 28 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXX.X...............................] duration: 204.316724ms FAIL (2025-08-07 22:07:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 204.337954ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX...............................] duration: 1.684843693s FAIL (2025-08-07 22:07:42 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.684878459s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXX..X..................................] duration: 588.092175ms FAIL (2025-08-07 22:07:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 (5h14m28.062269058s) [upload=5388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:25.266685321 (5h14m38.853741562s) [upload=5388]: Done uploading segments... Aug 7 21:41:25.266706391 (5h14m38.853762631s) [upload=5388]: Upload finished. err: <nil> Aug 7 21:41:41.070176076 (5h14m54.657232316s) [upload=5396]: Starting upload Aug 7 21:41:41.070246557 (5h14m54.657302798s) [upload=5396]: Done starting upload Aug 7 21:41:41.070459143 (5h14m54.657515385s) [upload=5396]: Uploading segments... Aug 7 21:41:41.070697238 (5h14m54.657753479s) [upload=5396]: Got next segment. Inline: false Aug 7 21:41:41.070732264 (5h14m54.657788505s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:41.070737293 (5h14m54.657793524s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:41:41.825834701 (5h14m55.412890922s) [seg_pos={0 0}, upload=5396]: Begin upload segment... Aug 7 21:41:41.825865027 (5h14m55.412921237s) [seg_pos={0 0}, upload=5396]: Done begin upload segment. Aug 7 21:41:41.825886015 (5h14m55.412942237s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.825988818 (5h14m55.413045068s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826203979 (5h14m55.413260201s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826435742 (5h14m55.413491953s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826541530 (5h14m55.41359777s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826653037 (5h14m55.413709248s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826875332 (5h14m55.413931543s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826921499 (5h14m55.41397774s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827091606 (5h14m55.414147827s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827323819 (5h14m55.41438003s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827393500 (5h14m55.414449742s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827542938 (5h14m55.414599159s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827778078 (5h14m55.414834318s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827801491 (5h14m55.414857712s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828365735 (5h14m55.415421976s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828913136 (5h14m55.415969378s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.934607129 (5h14m55.52166334s) [upload=5396]: Next returned nil segment Aug 7 21:41:41.934619282 (5h14m55.521675503s) [upload=5396]: Waiting for error group managing segments... Aug 7 21:41:41.945118551 (5h14m55.532174772s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.945901252 (5h14m55.532957473s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946161258 (5h14m55.533217498s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946386648 (5h14m55.533442869s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949024992 (5h14m55.536081243s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949218724 (5h14m55.536274945s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949657033 (5h14m55.536713244s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949683371 (5h14m55.536739612s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.950070354 (5h14m55.537126575s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.953623165 (5h14m55.540679407s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.957708261 (5h14m55.544764482s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964243811 (5h14m55.551300021s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964842018 (5h14m55.551898239s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964860783 (5h14m55.551917004s) [seg_pos={0 0}, upload=5396]: Segment reached optimal threshold of 13 pieces. Aug 7 21:41:41.964962292 (5h14m55.552018503s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965019739 (5h14m55.55207595s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965087696 (5h14m55.552143947s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965118783 (5h14m55.552175005s) [seg_pos={0 0}, upload=5396]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:41:41.965239138 (5h14m55.552295349s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:41:41.965243687 (5h14m55.552299898s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:46.509965896 (5h15m0.097022118s) [upload=5396]: Done uploading segments... Aug 7 21:41:46.510002955 (5h15m0.097059167s) [upload=5396]: Upload finished. err: <nil> Aug 7 21:42:01.196715658 (5h15m14.783771899s) [upload=5401]: Starting upload Aug 7 21:42:01.196842946 (5h15m14.783899187s) [upload=5401]: Done starting upload Aug 7 21:42:01.196915852 (5h15m14.783972103s) [upload=5401]: Uploading segments... Aug 7 21:42:01.197283067 (5h15m14.784339309s) [upload=5401]: Got next segment. Inline: false Aug 7 21:42:01.197355652 (5h15m14.784411984s) [upload=5401]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:01.197365230 (5h15m14.784421472s) [upload=5401]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:42:31.170606605 (5h15m44.757662826s) [upload=5401]: Next segment err: context deadline exceeded Aug 7 21:42:31.170727651 (5h15m44.757783872s) [upload=5401]: Done uploading segments... Aug 7 21:42:31.170749551 (5h15m44.757805773s) [upload=5401]: Upload finished. err: context deadline exceeded Duration:* 588.115448ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 20 out of 60 is failed (threshold=3) [.......XXXXXXXXXXXXXXXXX.XX......X..........................] duration: 580.318834ms FAIL (2025-08-07 22:07:33 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== .251202997s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:13.664150523 (5h20m27.251206734s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:57.268638109 (5h21m10.855694349s) [seg_pos={0 0}, upload=5487]: Begin upload segment... Aug 7 21:47:57.268679396 (5h21m10.855735637s) [seg_pos={0 0}, upload=5487]: Done begin upload segment. Aug 7 21:47:57.268708491 (5h21m10.855764731s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.268768562 (5h21m10.855824803s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269205608 (5h21m10.856261849s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269223562 (5h21m10.856279802s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269609953 (5h21m10.856666204s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269616095 (5h21m10.856672345s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269967370 (5h21m10.857023612s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269989231 (5h21m10.857045472s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270389678 (5h21m10.85744593s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270397674 (5h21m10.857453915s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270761794 (5h21m10.857818034s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270773264 (5h21m10.857829505s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271108780 (5h21m10.858165022s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271123869 (5h21m10.85818011s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271518095 (5h21m10.858574346s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272004884 (5h21m10.859061125s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272493695 (5h21m10.859549937s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.273356537 (5h21m10.860412828s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.396251337 (5h21m10.983307579s) [upload=5487]: Next returned nil segment Aug 7 21:47:57.396270474 (5h21m10.983326714s) [upload=5487]: Waiting for error group managing segments... Aug 7 21:47:57.416691513 (5h21m11.003747763s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.420507917 (5h21m11.007564157s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.425516195 (5h21m11.012572446s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.427782495 (5h21m11.014838736s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.431475158 (5h21m11.018531398s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.432679355 (5h21m11.019735596s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.449481709 (5h21m11.036537951s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.451301637 (5h21m11.038357878s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.469893221 (5h21m11.056949472s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.470298257 (5h21m11.057354497s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.475204656 (5h21m11.062260917s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.549461095 (5h21m11.136517316s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.661813247 (5h21m11.248869498s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755309651 (5h21m11.342365892s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755326954 (5h21m11.342383194s) [seg_pos={0 0}, upload=5487]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:57.755590967 (5h21m11.342647207s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755687266 (5h21m11.342743507s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUV64TP8cTQhBjswnhcWfQHwYEEWqr5ASuPjtEx3X6ZQNVhS5): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755708706 (5h21m11.342764947s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection Aug 7 21:47:57.755820425 (5h21m11.342876656s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WKWWwa3svATpoNY1z7LNvYmA8gA6ZFpJsk17G4c37tDJVJ2wj): protocol: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection Aug 7 21:47:57.755837257 (5h21m11.342893498s) [seg_pos={0 0}, upload=5487]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:57.755901476 (5h21m11.342957717s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:57.755906926 (5h21m11.342963168s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:45.449359965 (5h21m59.036416176s) [upload=5487]: Done uploading segments... Aug 7 21:48:45.449390482 (5h21m59.036446703s) [upload=5487]: Upload finished. err: metaclient: context deadline exceeded Duration: 580.349341ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 27 out of 60 is failed (threshold=3) [X.XXXXXXXXXXXXXXXXXXXXXXXXXX................................] duration: 1.563507863s FAIL (2025-08-07 22:07:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.563525926s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXX.XX....................................] duration: 519.104829ms FAIL (2025-08-07 22:07:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 519.125037ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXX..X.................................] duration: 338.108427ms FAIL (2025-08-07 22:07:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== 7 (5h14m28.062269058s) [upload=5388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:25.266685321 (5h14m38.853741562s) [upload=5388]: Done uploading segments... Aug 7 21:41:25.266706391 (5h14m38.853762631s) [upload=5388]: Upload finished. err: <nil> Aug 7 21:41:41.070176076 (5h14m54.657232316s) [upload=5396]: Starting upload Aug 7 21:41:41.070246557 (5h14m54.657302798s) [upload=5396]: Done starting upload Aug 7 21:41:41.070459143 (5h14m54.657515385s) [upload=5396]: Uploading segments... Aug 7 21:41:41.070697238 (5h14m54.657753479s) [upload=5396]: Got next segment. Inline: false Aug 7 21:41:41.070732264 (5h14m54.657788505s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:41.070737293 (5h14m54.657793524s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:41:41.825834701 (5h14m55.412890922s) [seg_pos={0 0}, upload=5396]: Begin upload segment... Aug 7 21:41:41.825865027 (5h14m55.412921237s) [seg_pos={0 0}, upload=5396]: Done begin upload segment. Aug 7 21:41:41.825886015 (5h14m55.412942237s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.825988818 (5h14m55.413045068s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826203979 (5h14m55.413260201s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826435742 (5h14m55.413491953s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826541530 (5h14m55.41359777s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826653037 (5h14m55.413709248s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826875332 (5h14m55.413931543s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.826921499 (5h14m55.41397774s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827091606 (5h14m55.414147827s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827323819 (5h14m55.41438003s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827393500 (5h14m55.414449742s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827542938 (5h14m55.414599159s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827778078 (5h14m55.414834318s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.827801491 (5h14m55.414857712s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828365735 (5h14m55.415421976s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.828913136 (5h14m55.415969378s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Uploading piece... Aug 7 21:41:41.934607129 (5h14m55.52166334s) [upload=5396]: Next returned nil segment Aug 7 21:41:41.934619282 (5h14m55.521675503s) [upload=5396]: Waiting for error group managing segments... Aug 7 21:41:41.945118551 (5h14m55.532174772s) [noise=true , address=45.91.116.4:4940 , piece_id=ETERKOUZIHEOKPCAJPSUBGXFMDLKEELCVMIW34LR7G66FBFMUNTA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.945901252 (5h14m55.532957473s) [noise=true , address=45.91.116.8:4048 , piece_id=XZNUB5SDPWSYVGHGMFFQZ4BZMBP2GMKF76GXWQ2MXAGT42I6I4FQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946161258 (5h14m55.533217498s) [noise=true , address=45.91.116.5:4298 , piece_id=C3BQRSMSRSM4FJ7SWEBPHH7UKNUJWCDD5AEEBEAYN6OMCM2VRSWA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.946386648 (5h14m55.533442869s) [noise=true , address=77.90.31.12:4592 , piece_id=75OFHQZRJKZYL3QJ37T6QLDED663WXIEFAHTH2BQZLZ46PY5YPLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949024992 (5h14m55.536081243s) [noise=true , address=45.91.116.5:4299 , piece_id=XY3RW62XNQVLM6NTAZTBF26HWTY5KR3ZEJGVYTSTENY2B77YLBMA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949218724 (5h14m55.536274945s) [noise=true , address=45.91.116.4:4937 , piece_id=BYNKX2DGOGITTOYV7IH73WDLAPXHXWG5V5WFIJDTM5RUJQYP6MLQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949657033 (5h14m55.536713244s) [noise=true , address=77.90.31.13:4627 , piece_id=7MG24AD6AUVL6X66RZJ43OSAM7FLBB33KQY4ENE4Q3NR6PA63YQQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.949683371 (5h14m55.536739612s) [noise=true , address=104.36.82.2:4732 , piece_id=2HK2G5XCNE6UN3COUOG6V4IVCLTRVEMNNCQMV5SZLXRFTO2FJUYQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.950070354 (5h14m55.537126575s) [noise=true , address=104.36.82.11:4114 , piece_id=KB2SDFXXBJDEYUHUNIO3YSANNDXVREF46SGOSWQ3YBHRR42TFD4A, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.953623165 (5h14m55.540679407s) [noise=true , address=104.36.82.6:4875 , piece_id=B263E3OQZQDAAYHN3YIPJ4RBLTNOFNOXAZWYTVEGGVRM2YES3PBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.957708261 (5h14m55.544764482s) [noise=true , address=104.36.82.5:4837 , piece_id=32VX7X2CPAQKJGKQYNDUF2U6GPAWZKEWYALRTRSFC6TPIQJBBXCQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964243811 (5h14m55.551300021s) [noise=true , address=77.90.31.10:4519 , piece_id=OQZPQPV2NUVKLMSHONEMH2HUK52KXL2E6NXXSO2J6CP7IDTRVT7Q, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964842018 (5h14m55.551898239s) [noise=true , address=77.90.31.14:4661 , piece_id=U3YYDMWVGG4NEV5OSK4ATJ2RNTEYCIXKH3KXY7UR3RGJXEUA25IA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: <nil> Aug 7 21:41:41.964860783 (5h14m55.551917004s) [seg_pos={0 0}, upload=5396]: Segment reached optimal threshold of 13 pieces. Aug 7 21:41:41.964962292 (5h14m55.552018503s) [noise=true , address=77.90.31.17:4771 , piece_id=IYUYS3YHOD44ZF2OGKQX7JCFVW2GBU5S2TVC3FZ2M3JNF67TP4CQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965019739 (5h14m55.55207595s) [noise=true , address=104.36.82.10:4077 , piece_id=LZBNWOC4E35SX6FKBLATK2QFZDUWA7CBA5RY422YHVMMKO5XDSBQ, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965087696 (5h14m55.552143947s) [noise=true , address=104.36.82.7:4911 , piece_id=GLWOEWKACQVRHXQ6EY45IYR5N7VCP75FUZIYT5WRQG2VXULRUIUA, seg_pos={0 0}, upload=5396]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Aug 7 21:41:41.965118783 (5h14m55.552175005s) [seg_pos={0 0}, upload=5396]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:41:41.965239138 (5h14m55.552295349s) [upload=5396]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:41:41.965243687 (5h14m55.552299898s) [upload=5396]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:46.509965896 (5h15m0.097022118s) [upload=5396]: Done uploading segments... Aug 7 21:41:46.510002955 (5h15m0.097059167s) [upload=5396]: Upload finished. err: <nil> Aug 7 21:42:01.196715658 (5h15m14.783771899s) [upload=5401]: Starting upload Aug 7 21:42:01.196842946 (5h15m14.783899187s) [upload=5401]: Done starting upload Aug 7 21:42:01.196915852 (5h15m14.783972103s) [upload=5401]: Uploading segments... Aug 7 21:42:01.197283067 (5h15m14.784339309s) [upload=5401]: Got next segment. Inline: false Aug 7 21:42:01.197355652 (5h15m14.784411984s) [upload=5401]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:01.197365230 (5h15m14.784421472s) [upload=5401]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:42:31.170606605 (5h15m44.757662826s) [upload=5401]: Next segment err: context deadline exceeded Aug 7 21:42:31.170727651 (5h15m44.757783872s) [upload=5401]: Done uploading segments... Aug 7 21:42:31.170749551 (5h15m44.757805773s) [upload=5401]: Upload finished. err: context deadline exceeded Duration:* 338.129365ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [......XX...X................................................] duration: 20.84573ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:07:13 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 20.876367ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 35 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.XXXXX........................] duration: 172.019058ms FAIL (2025-08-07 22:07:12 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== shed. err: metaclient: context deadline exceeded Aug 7 21:38:26.513648977 (5h11m40.100705218s) [upload=5344]: Starting upload Aug 7 21:38:26.513731310 (5h11m40.100787552s) [upload=5344]: Done starting upload Aug 7 21:38:26.513792245 (5h11m40.100848496s) [upload=5344]: Uploading segments... Aug 7 21:38:26.513834112 (5h11m40.100890354s) [upload=5344]: Got next segment. Inline: true Aug 7 21:38:26.513839653 (5h11m40.100895904s) [upload=5344]: Waiting for error group managing segments... Aug 7 21:38:26.513884006 (5h11m40.100940247s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:26.513889947 (5h11m40.100946188s) [upload=5344]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:26.513893434 (5h11m40.100949675s) [upload=5344]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:38:56.502052245 (5h12m10.089108537s) [upload=5344]: Done uploading segments... Aug 7 21:38:56.502102730 (5h12m10.089158972s) [upload=5344]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:24.008006172 (5h12m37.595062394s) [upload=5359]: Starting upload Aug 7 21:39:24.008059211 (5h12m37.595115422s) [upload=5359]: Done starting upload Aug 7 21:39:24.008103604 (5h12m37.595159835s) [upload=5359]: Uploading segments... Aug 7 21:39:24.008156191 (5h12m37.595212413s) [upload=5359]: Got next segment. Inline: true Aug 7 21:39:24.008162193 (5h12m37.595218414s) [upload=5359]: Waiting for error group managing segments... Aug 7 21:39:24.008207327 (5h12m37.595263548s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:24.008212668 (5h12m37.595268878s) [upload=5359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:24.008216064 (5h12m37.595272285s) [upload=5359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:53.997226503 (5h13m7.584282744s) [upload=5359]: Done uploading segments... Aug 7 21:39:53.997270334 (5h13m7.584326586s) [upload=5359]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:36.779619947 (5h13m50.366676188s) [upload=5383]: Starting upload Aug 7 21:40:36.779696139 (5h13m50.36675238s) [upload=5383]: Done starting upload Aug 7 21:40:36.779752775 (5h13m50.366809016s) [upload=5383]: Uploading segments... Aug 7 21:40:36.779788803 (5h13m50.366845044s) [upload=5383]: Got next segment. Inline: true Aug 7 21:40:36.779794423 (5h13m50.366850655s) [upload=5383]: Waiting for error group managing segments... Aug 7 21:40:36.779835950 (5h13m50.366892191s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:36.779841010 (5h13m50.366897241s) [upload=5383]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:36.779844195 (5h13m50.366900437s) [upload=5383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:06.765292852 (5h14m20.352349064s) [upload=5383]: Done uploading segments... Aug 7 21:41:06.765314052 (5h14m20.352370273s) [upload=5383]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:47.961322510 (5h15m1.548378802s) [upload=5399]: Starting upload Aug 7 21:41:47.961442915 (5h15m1.548499166s) [upload=5399]: Done starting upload Aug 7 21:41:47.961510201 (5h15m1.548566443s) [upload=5399]: Uploading segments... Aug 7 21:41:47.961591062 (5h15m1.548647313s) [upload=5399]: Got next segment. Inline: true Aug 7 21:41:47.961599218 (5h15m1.548655458s) [upload=5399]: Waiting for error group managing segments... Aug 7 21:41:47.961647759 (5h15m1.548703999s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:47.961653760 (5h15m1.548710001s) [upload=5399]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:47.961657546 (5h15m1.548713787s) [upload=5399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:17.948677724 (5h15m31.535733945s) [upload=5399]: Done uploading segments... Aug 7 21:42:17.948691099 (5h15m31.53574731s) [upload=5399]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:18.081006705 (5h15m31.668062915s) [upload=5405]: Starting upload Aug 7 21:42:18.081065655 (5h15m31.668121866s) [upload=5405]: Done starting upload Aug 7 21:42:18.081113425 (5h15m31.668169635s) [upload=5405]: Uploading segments... Aug 7 21:42:18.081164870 (5h15m31.668221091s) [upload=5405]: Got next segment. Inline: true Aug 7 21:42:18.081169459 (5h15m31.66822568s) [upload=5405]: Waiting for error group managing segments... Aug 7 21:42:18.081201809 (5h15m31.66825803s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:18.081205546 (5h15m31.668261757s) [upload=5405]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:18.081208271 (5h15m31.668264483s) [upload=5405]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:18.161269758 (5h15m31.748325999s) [upload=5405]: Done uploading segments... Aug 7 21:42:18.161287411 (5h15m31.748343652s) [upload=5405]: Upload finished. err: <nil> Aug 7 21:43:12.878567983 (5h16m26.465624204s) [upload=5420]: Starting upload Aug 7 21:43:12.878678820 (5h16m26.465735041s) [upload=5420]: Done starting upload Aug 7 21:43:12.878731839 (5h16m26.46578806s) [upload=5420]: Uploading segments... Aug 7 21:43:12.878762908 (5h16m26.465819129s) [upload=5420]: Got next segment. Inline: true Aug 7 21:43:12.878766635 (5h16m26.465822856s) [upload=5420]: Waiting for error group managing segments... Aug 7 21:43:12.878799204 (5h16m26.465855425s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:12.878803332 (5h16m26.465859543s) [upload=5420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:12.878805566 (5h16m26.465861787s) [upload=5420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:42.863779226 (5h16m56.450835478s) [upload=5420]: Done uploading segments... Aug 7 21:43:42.863802450 (5h16m56.450858691s) [upload=5420]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:51.464096058 (5h17m5.051152299s) [upload=5431]: Starting upload Aug 7 21:43:51.464210432 (5h17m5.051266673s) [upload=5431]: Done starting upload Aug 7 21:43:51.464274041 (5h17m5.051330282s) [upload=5431]: Uploading segments... Aug 7 21:43:51.464330065 (5h17m5.051386366s) [upload=5431]: Got next segment. Inline: true Aug 7 21:43:51.464343630 (5h17m5.051399912s) [upload=5431]: Waiting for error group managing segments... Aug 7 21:43:51.464415315 (5h17m5.051471556s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:51.464424673 (5h17m5.051480923s) [upload=5431]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:51.464431204 (5h17m5.051487455s) [upload=5431]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:44:21.448488824 (5h17m35.035545036s) [upload=5431]: Done uploading segments... Aug 7 21:44:21.448528209 (5h17m35.03558443s) [upload=5431]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:44:50.396194570 (5h18m3.983250782s) [upload=5444]: Starting upload Aug 7 21:44:50.396243653 (5h18m3.983299864s) [upload=5444]: Done starting upload Aug 7 21:44:50.396286463 (5h18m3.983342684s) [upload=5444]: Uploading segments... Aug 7 21:44:50.396309054 (5h18m3.983365266s) [upload=5444]: Got next segment. Inline: true Aug 7 21:44:50.396312381 (5h18m3.983368592s) [upload=5444]: Waiting for error group managing segments... Aug 7 21:44:50.396342687 (5h18m3.983398908s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:50.396346635 (5h18m3.983402856s) [upload=5444]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:50.396348889 (5h18m3.98340511s) [upload=5444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:20.384461417 (5h18m33.971517668s) [upload=5444]: Done uploading segments... Aug 7 21:45:20.384482547 (5h18m33.971538788s) [upload=5444]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:58.414522311 (5h19m12.001578522s) [upload=5465]: Starting upload Aug 7 21:45:58.414582292 (5h19m12.001638504s) [upload=5465]: Done starting upload Aug 7 21:45:58.414634661 (5h19m12.001690882s) [upload=5465]: Uploading segments... Aug 7 21:45:58.414662251 (5h19m12.001718462s) [upload=5465]: Got next segment. Inline: true Aug 7 21:45:58.414665918 (5h19m12.00172213s) [upload=5465]: Waiting for error group managing segments... Aug 7 21:45:58.414697247 (5h19m12.001753468s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:58.414700753 (5h19m12.001756975s) [upload=5465]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:58.414703007 (5h19m12.001759229s) [upload=5465]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:46:28.397843251 (5h19m41.984899492s) [upload=5465]: Done uploading segments... Aug 7 21:46:28.397885870 (5h19m41.984942112s) [upload=5465]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:07.964355881 (5h20m21.551412133s) [upload=5484]: Starting upload Aug 7 21:47:07.964442332 (5h20m21.551498584s) [upload=5484]: Done starting upload Aug 7 21:47:07.964529184 (5h20m21.551585426s) [upload=5484]: Uploading segments... Aug 7 21:47:07.964568658 (5h20m21.5516249s) [upload=5484]: Got next segment. Inline: true Aug 7 21:47:07.964573968 (5h20m21.551630209s) [upload=5484]: Waiting for error group managing segments... Aug 7 21:47:07.964617630 (5h20m21.551673881s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:07.964623640 (5h20m21.551679882s) [upload=5484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:07.964628048 (5h20m21.55168429s) [upload=5484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:37.944795042 (5h20m51.531851283s) [upload=5484]: Done uploading segments... Aug 7 21:47:37.944820539 (5h20m51.531876781s) [upload=5484]: Upload finished. err: metaclient: context deadline exceeded Duration: 172.04684ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 27 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXX...X.....X.........................] duration: 269.369216ms FAIL (2025-08-07 22:07:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 269.391837ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 29 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXX.X..............................] duration: 204.709892ms FAIL (2025-08-07 22:07:07 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 204.735018ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [XX.XXXXXXXXXXXXXXXXXXXXXXXXXX...............................] duration: 1.497263318s FAIL (2025-08-07 22:07:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.497288354s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 31 out of 60 is failed (threshold=3) [.XXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.XX..........................] duration: 175.196129ms FAIL (2025-08-07 22:06:56 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 31725988s) [upload=5324]: Upload finished. err: <nil> Aug 7 21:38:15.287715124 (5h11m28.874771365s) [upload=5341]: Starting upload Aug 7 21:38:15.287780276 (5h11m28.874836516s) [upload=5341]: Done starting upload Aug 7 21:38:15.287844856 (5h11m28.874901107s) [upload=5341]: Uploading segments... Aug 7 21:38:15.287889961 (5h11m28.874946201s) [upload=5341]: Got next segment. Inline: true Aug 7 21:38:15.287895600 (5h11m28.874951842s) [upload=5341]: Waiting for error group managing segments... Aug 7 21:38:15.287984066 (5h11m28.875040317s) [upload=5341]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:38:15.287990077 (5h11m28.875046318s) [upload=5341]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:38:15.287993564 (5h11m28.875049804s) [upload=5341]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:39:15.275679218 (5h12m28.862735459s) [upload=5341]: Done uploading segments... Aug 7 21:39:15.275708622 (5h12m28.862764883s) [upload=5341]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:39:15.358605080 (5h12m28.945661322s) [upload=5358]: Starting upload Aug 7 21:39:15.358690641 (5h12m28.945746882s) [upload=5358]: Done starting upload Aug 7 21:39:15.358748869 (5h12m28.945805111s) [upload=5358]: Uploading segments... Aug 7 21:39:15.358790938 (5h12m28.94584718s) [upload=5358]: Got next segment. Inline: true Aug 7 21:39:15.358796779 (5h12m28.94585302s) [upload=5358]: Waiting for error group managing segments... Aug 7 21:39:15.358840480 (5h12m28.945896721s) [upload=5358]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:39:15.358846441 (5h12m28.945902683s) [upload=5358]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:39:15.358849918 (5h12m28.945906159s) [upload=5358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:40:15.344303433 (5h13m28.931359654s) [upload=5358]: Done uploading segments... Aug 7 21:40:15.344323120 (5h13m28.931379331s) [upload=5358]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:40:15.438334176 (5h13m29.025390397s) [upload=5373]: Starting upload Aug 7 21:40:15.438379379 (5h13m29.025435601s) [upload=5373]: Done starting upload Aug 7 21:40:15.438418503 (5h13m29.025474714s) [upload=5373]: Uploading segments... Aug 7 21:40:15.438447206 (5h13m29.025503427s) [upload=5373]: Got next segment. Inline: true Aug 7 21:40:15.438450683 (5h13m29.025506894s) [upload=5373]: Waiting for error group managing segments... Aug 7 21:40:15.438481920 (5h13m29.025538141s) [upload=5373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:40:15.438487511 (5h13m29.025543722s) [upload=5373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:40:15.438489826 (5h13m29.025546047s) [upload=5373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:41:15.423387101 (5h14m29.010443342s) [upload=5373]: Done uploading segments... Aug 7 21:41:15.423408011 (5h14m29.010464251s) [upload=5373]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:41:15.514709386 (5h14m29.101765628s) [upload=5390]: Starting upload Aug 7 21:41:15.514790870 (5h14m29.10184711s) [upload=5390]: Done starting upload Aug 7 21:41:15.514852845 (5h14m29.101909086s) [upload=5390]: Uploading segments... Aug 7 21:41:15.514892409 (5h14m29.10194865s) [upload=5390]: Got next segment. Inline: true Aug 7 21:41:15.514898240 (5h14m29.101954481s) [upload=5390]: Waiting for error group managing segments... Aug 7 21:41:15.514941951 (5h14m29.101998192s) [upload=5390]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:41:15.514947551 (5h14m29.102003793s) [upload=5390]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:41:15.514951138 (5h14m29.102007389s) [upload=5390]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:42:15.497042181 (5h15m29.084098402s) [upload=5390]: Done uploading segments... Aug 7 21:42:15.497058301 (5h15m29.084114512s) [upload=5390]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:42:15.596278951 (5h15m29.183335172s) [upload=5404]: Starting upload Aug 7 21:42:15.596337329 (5h15m29.183393541s) [upload=5404]: Done starting upload Aug 7 21:42:15.596382114 (5h15m29.183438324s) [upload=5404]: Uploading segments... Aug 7 21:42:15.596411919 (5h15m29.18346814s) [upload=5404]: Got next segment. Inline: true Aug 7 21:42:15.596415937 (5h15m29.183472158s) [upload=5404]: Waiting for error group managing segments... Aug 7 21:42:15.596447405 (5h15m29.183503626s) [upload=5404]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:42:15.596451042 (5h15m29.183507263s) [upload=5404]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:42:15.596453266 (5h15m29.183509487s) [upload=5404]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:15.584735444 (5h16m29.171791655s) [upload=5404]: Done uploading segments... Aug 7 21:43:15.584747787 (5h16m29.171804008s) [upload=5404]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:43:22.691047415 (5h16m36.278103646s) [upload=5423]: Starting upload Aug 7 21:43:22.691110724 (5h16m36.278166955s) [upload=5423]: Done starting upload Aug 7 21:43:22.691177108 (5h16m36.278233338s) [upload=5423]: Uploading segments... Aug 7 21:43:22.691210019 (5h16m36.27826625s) [upload=5423]: Got next segment. Inline: true Aug 7 21:43:22.691214187 (5h16m36.278270418s) [upload=5423]: Waiting for error group managing segments... Aug 7 21:43:22.691250704 (5h16m36.278306925s) [upload=5423]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:43:22.691255263 (5h16m36.278311494s) [upload=5423]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:43:22.691258188 (5h16m36.278314419s) [upload=5423]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:43:22.751511620 (5h16m36.338567872s) [upload=5423]: Done uploading segments... Aug 7 21:43:22.751530204 (5h16m36.338586455s) [upload=5423]: Upload finished. err: <nil> Aug 7 21:44:13.185700222 (5h17m26.772756433s) [upload=5437]: Starting upload Aug 7 21:44:13.185766515 (5h17m26.772822736s) [upload=5437]: Done starting upload Aug 7 21:44:13.185811771 (5h17m26.772867981s) [upload=5437]: Uploading segments... Aug 7 21:44:13.185877192 (5h17m26.772933443s) [upload=5437]: Got next segment. Inline: true Aug 7 21:44:13.185884005 (5h17m26.772940226s) [upload=5437]: Waiting for error group managing segments... Aug 7 21:44:13.185926104 (5h17m26.772982314s) [upload=5437]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:44:13.185931043 (5h17m26.772987254s) [upload=5437]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:44:13.185934560 (5h17m26.77299077s) [upload=5437]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:13.172887995 (5h18m26.759944236s) [upload=5437]: Done uploading segments... Aug 7 21:45:13.172908133 (5h18m26.759964374s) [upload=5437]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:45:13.256441576 (5h18m26.843497827s) [upload=5449]: Starting upload Aug 7 21:45:13.256514653 (5h18m26.843570894s) [upload=5449]: Done starting upload Aug 7 21:45:13.256603699 (5h18m26.84365995s) [upload=5449]: Uploading segments... Aug 7 21:45:13.256644835 (5h18m26.843701076s) [upload=5449]: Got next segment. Inline: true Aug 7 21:45:13.256650195 (5h18m26.843706436s) [upload=5449]: Waiting for error group managing segments... Aug 7 21:45:13.256695270 (5h18m26.843751511s) [upload=5449]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:45:13.256701020 (5h18m26.843757262s) [upload=5449]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:45:13.256704657 (5h18m26.843760898s) [upload=5449]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:45:13.340684827 (5h18m26.927741079s) [upload=5449]: Done uploading segments... Aug 7 21:45:13.340709023 (5h18m26.927765264s) [upload=5449]: Upload finished. err: <nil> Aug 7 21:46:18.644805531 (5h19m32.231861771s) [upload=5472]: Starting upload Aug 7 21:46:18.644902892 (5h19m32.231959133s) [upload=5472]: Done starting upload Aug 7 21:46:18.644968324 (5h19m32.232024565s) [upload=5472]: Uploading segments... Aug 7 21:46:18.645015051 (5h19m32.232071292s) [upload=5472]: Got next segment. Inline: true Aug 7 21:46:18.645020682 (5h19m32.232076933s) [upload=5472]: Waiting for error group managing segments... Aug 7 21:46:18.645064344 (5h19m32.232120585s) [upload=5472]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:46:18.645070025 (5h19m32.232126266s) [upload=5472]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:46:18.645073771 (5h19m32.232130013s) [upload=5472]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:47:18.630338668 (5h20m32.217394879s) [upload=5472]: Done uploading segments... Aug 7 21:47:18.630353927 (5h20m32.217410137s) [upload=5472]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:47:18.710984265 (5h20m32.298040506s) [upload=5489]: Starting upload Aug 7 21:47:18.711056571 (5h20m32.298112812s) [upload=5489]: Done starting upload Aug 7 21:47:18.711125208 (5h20m32.29818145s) [upload=5489]: Uploading segments... Aug 7 21:47:18.711173729 (5h20m32.29822996s) [upload=5489]: Got next segment. Inline: true Aug 7 21:47:18.711179229 (5h20m32.298235471s) [upload=5489]: Waiting for error group managing segments... Aug 7 21:47:18.711220847 (5h20m32.298277087s) [upload=5489]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:18.711225816 (5h20m32.298282057s) [upload=5489]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:47:18.711229112 (5h20m32.298285354s) [upload=5489]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:18.698417946 (5h21m32.285474187s) [upload=5489]: Done uploading segments... Aug 7 21:48:18.698439776 (5h21m32.285496017s) [upload=5489]: Upload finished. err: metaclient: context deadline exceeded Duration: 175.215155ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.............X...X...X......................................] duration: 118.105461ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:06:46 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 118.123545ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.X.............................] duration: 253.781197ms FAIL (2025-08-07 22:06:41 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 253.802327ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.......XX...X...............................................] duration: 25.416243ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-07 22:06:38 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 25.45239ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 30 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX..............................] duration: 1.652752423s FAIL (2025-08-07 22:06:35 UTC) operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.652780585s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 20 out of 60 is failed (threshold=3) [........XXXXXXXXXXXXXXXXX.XX......X.........................] duration: 640.770075ms FAIL (2025-08-07 22:06:28 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== .251202997s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:47:13.664150523 (5h20m27.251206734s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:47:57.268638109 (5h21m10.855694349s) [seg_pos={0 0}, upload=5487]: Begin upload segment... Aug 7 21:47:57.268679396 (5h21m10.855735637s) [seg_pos={0 0}, upload=5487]: Done begin upload segment. Aug 7 21:47:57.268708491 (5h21m10.855764731s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.268768562 (5h21m10.855824803s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269205608 (5h21m10.856261849s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269223562 (5h21m10.856279802s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269609953 (5h21m10.856666204s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269616095 (5h21m10.856672345s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269967370 (5h21m10.857023612s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.269989231 (5h21m10.857045472s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270389678 (5h21m10.85744593s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270397674 (5h21m10.857453915s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270761794 (5h21m10.857818034s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.270773264 (5h21m10.857829505s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271108780 (5h21m10.858165022s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271123869 (5h21m10.85818011s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.271518095 (5h21m10.858574346s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272004884 (5h21m10.859061125s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.272493695 (5h21m10.859549937s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.273356537 (5h21m10.860412828s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Uploading piece... Aug 7 21:47:57.396251337 (5h21m10.983307579s) [upload=5487]: Next returned nil segment Aug 7 21:47:57.396270474 (5h21m10.983326714s) [upload=5487]: Waiting for error group managing segments... Aug 7 21:47:57.416691513 (5h21m11.003747763s) [noise=true , address=104.36.82.60:4366 , piece_id=K5AFATILGWENKT7RCXTSZA2NXEZNRHVURXKGKSBYY5CMXRSYJ3IQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.420507917 (5h21m11.007564157s) [noise=true , address=45.91.116.9:4266 , piece_id=Z446OBO7XDWRNFYB3LOLGEUNXOBUDHBFUJLDLUJIYNKCBZMFT6UQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.425516195 (5h21m11.012572446s) [noise=true , address=104.36.82.23:4453 , piece_id=LWBE3IKOWYAS6LLSYJKAMR7UQAXMKA6YRHKYB2AD5C25FATIIO6Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.427782495 (5h21m11.014838736s) [noise=true , address=104.36.82.58:4271 , piece_id=BASVAJCXQU5HSUZOF5FTYAKAIUAQ3HM7LKDL2FQIFNMPWTZ2TF3A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.431475158 (5h21m11.018531398s) [noise=true , address=104.36.82.36:4650 , piece_id=ZFUHFXUS5EUMSYZRMH47P67BYRBR4NSI7QJZBKN2BJCXDU4HAK3Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.432679355 (5h21m11.019735596s) [noise=true , address=104.36.82.43:4867 , piece_id=BKZQ3IXGP5L3HYUI7KIDAKUOQEHC5SW4J56GJ65TMSURYKZHBG4A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.449481709 (5h21m11.036537951s) [noise=true , address=104.36.82.2:4733 , piece_id=ZSMLAY5EQYI6SJLJCXC6AX6BVWLAV34JEOT5OQHRCWQIZIAAZNSQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.451301637 (5h21m11.038357878s) [noise=true , address=104.36.82.55:4197 , piece_id=SVAL3VF2GDN3VAMCEBUJ7654GTZWP7VFECPPN5LVQPNI4UXAVLTA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.469893221 (5h21m11.056949472s) [noise=true , address=77.90.31.19:4871 , piece_id=Q6F3Q5WGIDQVVNLZ5B6UUXSMO4UOFQODSLHCGWO3OYEIDO4DLCOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.470298257 (5h21m11.057354497s) [noise=true , address=104.36.82.17:4331 , piece_id=W2HPBENUQT4QVMKRCRH34GNPHACSWJF4Y3OXUFP23GWDFU7YQ2MQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.475204656 (5h21m11.062260917s) [noise=true , address=104.36.82.47:4534 , piece_id=NZP6J3UBI7NTB7QNE3IBOSWUUD2ALPP4N7SQ5KAJYYKBUXCF4CJA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.549461095 (5h21m11.136517316s) [noise=true , address=45.91.116.6:4996 , piece_id=U6Z2Z4JE57FTIIPNYVVVEGFJ5OIIGFXHKVEAJCMXOE5AHY5KC53Q, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.661813247 (5h21m11.248869498s) [noise=true , address=104.36.82.37:4669 , piece_id=WDFHFMPCCKI42POS27ERI6VLIU4LNJR757YMFCYAHUEKP3ULKBZQ, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755309651 (5h21m11.342365892s) [noise=true , address=79.127.205.233:4641 , piece_id=Q7QSC5VXPJD7VS7A4OB4GWWBAXAIK4FUWKKFISA3GFHZLVFMQ43A, seg_pos={0 0}, upload=5487]: Done uploading piece. err: <nil> Aug 7 21:47:57.755326954 (5h21m11.342383194s) [seg_pos={0 0}, upload=5487]: Segment reached optimal threshold of 14 pieces. Aug 7 21:47:57.755590967 (5h21m11.342647207s) [noise=true , address=104.36.82.45:4918 , piece_id=6YEE6UIVZ6TATEGKR4KV5HD3GSVF2KOYSMZATI2P25437QCAMMUA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755687266 (5h21m11.342743507s) [noise=true , address=64.49.13.27:4966 , piece_id=HPI2TRSWEVIAQJG6BPB6XJUUTEM5DDESCROR3Z2F3Q3WIIWKRKZA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gUV64TP8cTQhBjswnhcWfQHwYEEWqr5ASuPjtEx3X6ZQNVhS5): protocol: expected piece hash; context canceled; EOF Aug 7 21:47:57.755708706 (5h21m11.342764947s) [noise=true , address=104.36.82.5:4858 , piece_id=MZWUW7TYF2AC2UAI7G3JY4A27WCPIR6PKYJESTVQDKYTEJWLZFOA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35426->104.36.82.5:4858: use of closed network connection Aug 7 21:47:57.755820425 (5h21m11.342876656s) [noise=true , address=104.36.82.61:4385 , piece_id=HSKW2KJ457X2FHEKLOLGPKWVW6OW67GDHGU7Y2JHEV34QNDDYKLA, seg_pos={0 0}, upload=5487]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WKWWwa3svATpoNY1z7LNvYmA8gA6ZFpJsk17G4c37tDJVJ2wj): protocol: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:55018->104.36.82.61:4385: use of closed network connection Aug 7 21:47:57.755837257 (5h21m11.342893498s) [seg_pos={0 0}, upload=5487]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:47:57.755901476 (5h21m11.342957717s) [upload=5487]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:47:57.755906926 (5h21m11.342963168s) [upload=5487]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:48:45.449359965 (5h21m59.036416176s) [upload=5487]: Done uploading segments... Aug 7 21:48:45.449390482 (5h21m59.036446703s) [upload=5487]: Upload finished. err: metaclient: context deadline exceeded Duration: 640.795634ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [..............X...X...X.....................................] duration: 128.345754ms remote: "tcp:136.0.77.2:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-07 22:06:21 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 128.365441ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXX.XX...................................] duration: 542.664879ms FAIL (2025-08-07 22:06:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 542.68175ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)