Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X...........................................X.X..XXX.....X.] duration: 1.865165054s FAIL (2025-08-08 00:03:12 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.865191384s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [X..X......................................XXX.X.XXX.X..XX.X.] duration: 188.853981ms FAIL (2025-08-08 00:03:12 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 188.872566ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.X..X......................................XXX.X.XXX.X..XX.X] duration: 30.001174502s FAIL (2025-08-08 00:03:08 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00121678s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X....................................................X....] duration: 233.469685ms FAIL (2025-08-08 00:03:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 233.494911ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [X..X............................................XXXXXXX...X.] duration: 1.544932048s FAIL (2025-08-08 00:03:03 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.545252967s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [XX..X............................................XXXXXXX...X] duration: 1m0.000407812s FAIL (2025-08-08 00:03:01 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000418542s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X...........................................X.X..XXX.....X] duration: 30.001517906s FAIL (2025-08-08 00:02:58 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001537984s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.X..........................................X...........X..] duration: 651.943877ms FAIL (2025-08-08 00:02:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 651.961869ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [...........................................XXXXXXXXX...X....] duration: 241.844949ms FAIL (2025-08-08 00:02:50 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 241.868032ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 177.817721ms FAIL (2025-08-08 00:02:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 177.971358ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X....XXX.X......................................XXXXX.....X.] duration: 503.327137ms FAIL (2025-08-08 00:02:43 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== . successful: 13 optimal: 13 errs: <nil> Aug 8 00:00:48.552480588 (7h34m2.13953683s) [upload=7750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:00:48.552487821 (7h34m2.139544062s) [upload=7750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:48.596293603 (7h34m2.183349845s) [upload=7750]: Done uploading segments... Aug 8 00:00:48.596306147 (7h34m2.183362389s) [upload=7750]: Upload finished. err: <nil> Aug 8 00:01:04.360824974 (7h34m17.947881195s) [upload=7753]: Starting upload Aug 8 00:01:04.360872293 (7h34m17.947928503s) [upload=7753]: Done starting upload Aug 8 00:01:04.360907538 (7h34m17.947963759s) [upload=7753]: Uploading segments... Aug 8 00:01:04.361092694 (7h34m17.948148915s) [upload=7753]: Got next segment. Inline: false Aug 8 00:01:04.361132418 (7h34m17.948188639s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:04.361138158 (7h34m17.94819437s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:01:04.384306121 (7h34m17.971362342s) [seg_pos={0 0}, upload=7753]: Begin upload segment... Aug 8 00:01:04.384337399 (7h34m17.97139362s) [seg_pos={0 0}, upload=7753]: Done begin upload segment. Aug 8 00:01:04.384352147 (7h34m17.971408358s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384363718 (7h34m17.97141994s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384394245 (7h34m17.971450456s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384420654 (7h34m17.971476876s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384672525 (7h34m17.971728766s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384680811 (7h34m17.971737021s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384893778 (7h34m17.971949989s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384905430 (7h34m17.97196164s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385111724 (7h34m17.972167945s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385321807 (7h34m17.972378028s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385547287 (7h34m17.972603499s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385780032 (7h34m17.972836253s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386010562 (7h34m17.973066783s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386253636 (7h34m17.973309857s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386512819 (7h34m17.97356903s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386763979 (7h34m17.9738202s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.481200832 (7h34m18.068257043s) [upload=7753]: Next returned nil segment Aug 8 00:01:04.481212825 (7h34m18.068269035s) [upload=7753]: Waiting for error group managing segments... Aug 8 00:01:04.493233591 (7h34m18.080289802s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494598709 (7h34m18.08165492s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494954183 (7h34m18.082010394s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.495060111 (7h34m18.082116332s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496576410 (7h34m18.083632621s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496907479 (7h34m18.083963689s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497177513 (7h34m18.084233724s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497538055 (7h34m18.084594266s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.499554820 (7h34m18.08661103s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501339521 (7h34m18.088395732s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501639971 (7h34m18.088696183s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.502941039 (7h34m18.089997261s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503363599 (7h34m18.0904198s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503371744 (7h34m18.090427954s) [seg_pos={0 0}, upload=7753]: Segment reached optimal threshold of 13 pieces. Aug 8 00:01:04.503387753 (7h34m18.090443955s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503507487 (7h34m18.090563698s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503586976 (7h34m18.090643187s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503596583 (7h34m18.090652794s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:01:04.503736805 (7h34m18.090793016s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:04.503742045 (7h34m18.090798256s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:04.558580310 (7h34m18.14563656s) [upload=7753]: Done uploading segments... Aug 8 00:01:04.558599866 (7h34m18.145656107s) [upload=7753]: Upload finished. err: <nil> Aug 8 00:02:02.017383460 (7h35m15.604439681s) [upload=7769]: Starting upload Aug 8 00:02:02.017467698 (7h35m15.604523899s) [upload=7769]: Done starting upload Aug 8 00:02:02.017580639 (7h35m15.604636861s) [upload=7769]: Uploading segments... Aug 8 00:02:02.017743643 (7h35m15.604799854s) [upload=7769]: Got next segment. Inline: false Aug 8 00:02:02.017795160 (7h35m15.60485137s) [upload=7769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:02.017800670 (7h35m15.604856881s) [upload=7769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:02:32.006066748 (7h35m45.593122959s) [upload=7769]: Next segment err: context deadline exceeded Aug 8 00:02:32.006146126 (7h35m45.593202338s) [upload=7769]: Done uploading segments... Aug 8 00:02:32.006171594 (7h35m45.593227815s) [upload=7769]: Upload finished. err: context deadline exceeded Duration: 503.355279ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X....XXX.X......................................XXXXX.....X] duration: 30.000979178s FAIL (2025-08-08 00:02:32 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== . successful: 13 optimal: 13 errs: <nil> Aug 8 00:00:48.552480588 (7h34m2.13953683s) [upload=7750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:00:48.552487821 (7h34m2.139544062s) [upload=7750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:48.596293603 (7h34m2.183349845s) [upload=7750]: Done uploading segments... Aug 8 00:00:48.596306147 (7h34m2.183362389s) [upload=7750]: Upload finished. err: <nil> Aug 8 00:01:04.360824974 (7h34m17.947881195s) [upload=7753]: Starting upload Aug 8 00:01:04.360872293 (7h34m17.947928503s) [upload=7753]: Done starting upload Aug 8 00:01:04.360907538 (7h34m17.947963759s) [upload=7753]: Uploading segments... Aug 8 00:01:04.361092694 (7h34m17.948148915s) [upload=7753]: Got next segment. Inline: false Aug 8 00:01:04.361132418 (7h34m17.948188639s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:01:04.361138158 (7h34m17.94819437s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:01:04.384306121 (7h34m17.971362342s) [seg_pos={0 0}, upload=7753]: Begin upload segment... Aug 8 00:01:04.384337399 (7h34m17.97139362s) [seg_pos={0 0}, upload=7753]: Done begin upload segment. Aug 8 00:01:04.384352147 (7h34m17.971408358s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384363718 (7h34m17.97141994s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384394245 (7h34m17.971450456s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384420654 (7h34m17.971476876s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384672525 (7h34m17.971728766s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384680811 (7h34m17.971737021s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384893778 (7h34m17.971949989s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.384905430 (7h34m17.97196164s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385111724 (7h34m17.972167945s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385321807 (7h34m17.972378028s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385547287 (7h34m17.972603499s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.385780032 (7h34m17.972836253s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386010562 (7h34m17.973066783s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386253636 (7h34m17.973309857s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386512819 (7h34m17.97356903s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.386763979 (7h34m17.9738202s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Uploading piece... Aug 8 00:01:04.481200832 (7h34m18.068257043s) [upload=7753]: Next returned nil segment Aug 8 00:01:04.481212825 (7h34m18.068269035s) [upload=7753]: Waiting for error group managing segments... Aug 8 00:01:04.493233591 (7h34m18.080289802s) [noise=true , address=45.91.116.5:4297 , piece_id=EAQ5OEBUIVZ3LTWIPLPJS2GRRZOMVW6JUUWJAJ7ORWLOPEZW7EMQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494598709 (7h34m18.08165492s) [noise=true , address=77.90.31.12:4590 , piece_id=U5DFPW5NOWT4DD6EYDQ2RJBVW7HDUPNWX7Z6MEGJG5WY6WXSTIDA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.494954183 (7h34m18.082010394s) [noise=true , address=77.90.31.17:4772 , piece_id=RZTHT5JSE6IQZ5DEWYOCZDHTELCCFJIC54ASDD75XXXUKJG6JYCA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.495060111 (7h34m18.082116332s) [noise=true , address=77.90.31.13:4627 , piece_id=5JEJYGFACMSTUQCH2XTIRXBWHSRATQSFFOOJDUGDZDAXUOZ5GL2A, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496576410 (7h34m18.083632621s) [noise=true , address=77.90.31.18:4807 , piece_id=3KCMVB4MNQOCY6ZJRKRSGL3ZVAFTCIZ2QEWLYEI5AO53IZOTVQGQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.496907479 (7h34m18.083963689s) [noise=true , address=45.91.116.9:4262 , piece_id=BBPREUY5UPTGGBNCRU5CYOFM7DE3CLLQLLOYCGBTQ6LYY445QJIQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497177513 (7h34m18.084233724s) [noise=true , address=104.36.82.13:4186 , piece_id=N34KWNZYJLBCFVY6EGH2AONNDORPJHB5WBPCYFKXV4TMODLSS7RA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.497538055 (7h34m18.084594266s) [noise=true , address=45.91.116.4:4939 , piece_id=ZJK37OBBPOLMXFNCAOJ4GWVSFOCUHOAWG4LZS6OZJDIKZBLIMQOA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.499554820 (7h34m18.08661103s) [noise=true , address=77.90.31.10:4518 , piece_id=UWYV7765UCFIE7A3BJIMBKUNCPRZLA35S2BTOVYXKDMHZ675ZDIA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501339521 (7h34m18.088395732s) [noise=true , address=104.36.82.14:4223 , piece_id=O4QA2LKOUM6ZKVENWWRRRNN7U4ZDFUN3RMGCPZHJYDN5TVCTB54Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.501639971 (7h34m18.088696183s) [noise=true , address=77.90.31.12:4589 , piece_id=ZR4Q5PPJLOT2M45N7BQM4TE2OS62CMFIJ735RKYPWRPCIFT3OQXA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.502941039 (7h34m18.089997261s) [noise=true , address=77.90.31.11:4553 , piece_id=T5SD2E7X2574ZIC2QSYSOLYX5722PIO6KEYJFG6H6COXYMYHFTVQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503363599 (7h34m18.0904198s) [noise=true , address=104.36.82.5:4838 , piece_id=PBLCVCMLKBUTLYB7JLKLLFD5B26S32EAQKDGOZTTQNLUBFRDJ3PQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503371744 (7h34m18.090427954s) [seg_pos={0 0}, upload=7753]: Segment reached optimal threshold of 13 pieces. Aug 8 00:01:04.503387753 (7h34m18.090443955s) [noise=true , address=104.36.82.15:4258 , piece_id=NVS7V2QTCB3O5T5WHV5NU3PXXPWZ7VM7WECCXI7SXDYEJNPRDAUQ, seg_pos={0 0}, upload=7753]: Done uploading piece. err: <nil> Aug 8 00:01:04.503507487 (7h34m18.090563698s) [noise=true , address=45.91.116.10:4081 , piece_id=4NVCB3RTBIDOBTPB2JTTRF2I3R6SEYCK6ICE2QJ7UNSV3G54IQHA, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503586976 (7h34m18.090643187s) [noise=true , address=104.36.82.7:4912 , piece_id=7EPXQBNQTEU2WOPLINAAYZFK62NYWGAENF4S3JU2LDBYTDIV3P6Q, seg_pos={0 0}, upload=7753]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Aug 8 00:01:04.503596583 (7h34m18.090652794s) [seg_pos={0 0}, upload=7753]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Aug 8 00:01:04.503736805 (7h34m18.090793016s) [upload=7753]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 00:01:04.503742045 (7h34m18.090798256s) [upload=7753]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:04.558580310 (7h34m18.14563656s) [upload=7753]: Done uploading segments... Aug 8 00:01:04.558599866 (7h34m18.145656107s) [upload=7753]: Upload finished. err: <nil> Aug 8 00:02:02.017383460 (7h35m15.604439681s) [upload=7769]: Starting upload Aug 8 00:02:02.017467698 (7h35m15.604523899s) [upload=7769]: Done starting upload Aug 8 00:02:02.017580639 (7h35m15.604636861s) [upload=7769]: Uploading segments... Aug 8 00:02:02.017743643 (7h35m15.604799854s) [upload=7769]: Got next segment. Inline: false Aug 8 00:02:02.017795160 (7h35m15.60485137s) [upload=7769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 00:02:02.017800670 (7h35m15.604856881s) [upload=7769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 00:02:32.006066748 (7h35m45.593122959s) [upload=7769]: Next segment err: context deadline exceeded Aug 8 00:02:32.006146126 (7h35m45.593202338s) [upload=7769]: Done uploading segments... Aug 8 00:02:32.006171594 (7h35m45.593227815s) [upload=7769]: Upload finished. err: context deadline exceeded Duration: 30.000997392s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 737.456795ms FAIL (2025-08-08 00:02:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 737.475931ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.X..........................................X...........X.] duration: 593.314537ms FAIL (2025-08-08 00:02:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 593.334164ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [......XXX.X.X.........................................XX..X.] duration: 28.294141937s FAIL (2025-08-08 00:02:15 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =7685]: Done starting upload Aug 7 23:56:26.112946671 (7h29m39.700002912s) [upload=7685]: Uploading segments... Aug 7 23:56:26.113307024 (7h29m39.700363265s) [upload=7685]: Got next segment. Inline: false Aug 7 23:56:26.113355876 (7h29m39.700412117s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:26.113362618 (7h29m39.700418859s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:41.416515391 (7h29m55.003571642s) [seg_pos={0 0}, upload=7685]: Begin upload segment... Aug 7 23:56:41.416597103 (7h29m55.003653344s) [seg_pos={0 0}, upload=7685]: Done begin upload segment. Aug 7 23:56:41.416637698 (7h29m55.003693939s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.416648018 (7h29m55.003704309s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417133103 (7h29m55.004189345s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417214976 (7h29m55.004271227s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417661369 (7h29m55.00471761s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417781764 (7h29m55.004838005s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418200426 (7h29m55.005256666s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418318265 (7h29m55.005374516s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418710699 (7h29m55.005766949s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418900132 (7h29m55.005956382s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419288596 (7h29m55.006344878s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419568088 (7h29m55.006624339s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419865433 (7h29m55.006921684s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420396514 (7h29m55.007452935s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420438683 (7h29m55.007494974s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.422079145 (7h29m55.009135386s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.423075114 (7h29m55.010131365s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.424123361 (7h29m55.011179632s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.543148900 (7h29m55.13020514s) [upload=7685]: Next returned nil segment Aug 7 23:56:41.543219551 (7h29m55.130275812s) [upload=7685]: Waiting for error group managing segments... Aug 7 23:56:41.555451953 (7h29m55.142508195s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.556831739 (7h29m55.14388798s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.558390238 (7h29m55.145446479s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.568256594 (7h29m55.155312844s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.579463192 (7h29m55.166519433s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.583078421 (7h29m55.170134671s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.584225170 (7h29m55.171281422s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.590665141 (7h29m55.177721381s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.611966581 (7h29m55.199022822s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.627727450 (7h29m55.214783701s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.645759177 (7h29m55.232815418s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.774571312 (7h29m55.361627554s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.876799657 (7h29m55.463855878s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000385207 (7h29m55.587441438s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000439059 (7h29m55.587495289s) [seg_pos={0 0}, upload=7685]: Segment reached optimal threshold of 14 pieces. Aug 7 23:56:42.000640734 (7h29m55.587696955s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000697911 (7h29m55.587754142s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dgidHuoyZFQZQyBAfQrBAn3FXjgePo8GjuzPyBijKp5xb4wk1): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000755489 (7h29m55.58781173s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AcLsSmVz9sRtuDLs47WoWYC66YSbr2aXBjzBdjayg22daQ2Hp): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000812144 (7h29m55.587868355s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fTtTieEUquaKNUNzqK98PkX7tVxjsiNeMLjz8kQJwdLne7d9R): protocol: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection Aug 7 23:56:42.000840427 (7h29m55.587896659s) [seg_pos={0 0}, upload=7685]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:56:42.000912191 (7h29m55.587968402s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:56:42.000916679 (7h29m55.5879729s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:47.368617454 (7h35m0.955673666s) [upload=7685]: Done uploading segments... Aug 8 00:01:47.368640257 (7h35m0.955696468s) [upload=7685]: Upload finished. err: metaclient: context deadline exceeded Duration: 28.294171742s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [..X..X......................................XXX.X.XXX.X..XX.] duration: 216.649791ms FAIL (2025-08-08 00:02:15 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 216.677382ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..X....................................................X...] duration: 222.974247ms FAIL (2025-08-08 00:02:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 223.000905ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.X...............................................XX.X.......] duration: 191.397549ms FAIL (2025-08-08 00:02:10 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Duration:* 191.423006ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X.X...........................................X.X..XXX.....] duration: 1.672497756s FAIL (2025-08-08 00:02:09 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.672521701s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [X..X..X......................................XXX.X.XXX.X..XX] duration: 30.000947251s FAIL (2025-08-08 00:02:02 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000968659s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 10 out of 60 is failed (threshold=3) [............................................XXXXXXXXX...X...] duration: 179.89504ms FAIL (2025-08-08 00:02:01 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 179.910209ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [XXX..X............................................XXXXXXX...] duration: 57.184568289s FAIL (2025-08-08 00:02:01 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 57.184586233s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.X..........................................X...........X] duration: 30.000505828s FAIL (2025-08-08 00:01:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000526206s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [.......XXX.X.X.........................................XX..X] duration: 5m21.270857701s FAIL (2025-08-08 00:01:47 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== =7685]: Done starting upload Aug 7 23:56:26.112946671 (7h29m39.700002912s) [upload=7685]: Uploading segments... Aug 7 23:56:26.113307024 (7h29m39.700363265s) [upload=7685]: Got next segment. Inline: false Aug 7 23:56:26.113355876 (7h29m39.700412117s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:26.113362618 (7h29m39.700418859s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:56:41.416515391 (7h29m55.003571642s) [seg_pos={0 0}, upload=7685]: Begin upload segment... Aug 7 23:56:41.416597103 (7h29m55.003653344s) [seg_pos={0 0}, upload=7685]: Done begin upload segment. Aug 7 23:56:41.416637698 (7h29m55.003693939s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.416648018 (7h29m55.003704309s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417133103 (7h29m55.004189345s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417214976 (7h29m55.004271227s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417661369 (7h29m55.00471761s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.417781764 (7h29m55.004838005s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418200426 (7h29m55.005256666s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418318265 (7h29m55.005374516s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418710699 (7h29m55.005766949s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.418900132 (7h29m55.005956382s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419288596 (7h29m55.006344878s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419568088 (7h29m55.006624339s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.419865433 (7h29m55.006921684s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420396514 (7h29m55.007452935s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.420438683 (7h29m55.007494974s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.422079145 (7h29m55.009135386s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.423075114 (7h29m55.010131365s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.424123361 (7h29m55.011179632s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Uploading piece... Aug 7 23:56:41.543148900 (7h29m55.13020514s) [upload=7685]: Next returned nil segment Aug 7 23:56:41.543219551 (7h29m55.130275812s) [upload=7685]: Waiting for error group managing segments... Aug 7 23:56:41.555451953 (7h29m55.142508195s) [noise=true , address=104.36.82.23:4473 , piece_id=EGPJ32OFTAEXMYOBCMRLAFZS6K3PWHZXPNHGDRBW3KH34FKCPFHA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.556831739 (7h29m55.14388798s) [noise=true , address=104.36.82.45:4918 , piece_id=UMF4G5VBSPQTQW2QEZUKJHD4LVTRONQ2SXJBU5Q3BJBP3ACNKYKA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.558390238 (7h29m55.145446479s) [noise=true , address=104.36.82.11:4144 , piece_id=FJ56K2U26BSF46WX7SW5XHGS73JRSZJ6RTVYERNXYAF3JXFJJNTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.568256594 (7h29m55.155312844s) [noise=true , address=104.36.82.43:4860 , piece_id=KVIAVLDP6S6SUL3LQIOELEXJPNCS6QF3VGMWLQRRVCX3IQSUT2GA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.579463192 (7h29m55.166519433s) [noise=true , address=104.36.82.53:4128 , piece_id=VWKV5IVSRDRHA5HGN7PIYMQM7YACXROD3DWWGFTDQSLC4HA6HT3A, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.583078421 (7h29m55.170134671s) [noise=true , address=45.91.116.11:4151 , piece_id=3QOIMFHI4YJY7KSKVVFZ73JNG5KJLXUV6YUSKWVBGLAEFCES7TRQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.584225170 (7h29m55.171281422s) [noise=true , address=104.36.82.9:4030 , piece_id=DGMZLB5RTQ3DVVXRWXR7YEGLFS3LAO5C3SRHZY3MHKOIFTVI66XQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.590665141 (7h29m55.177721381s) [noise=true , address=104.36.82.54:4166 , piece_id=LV7CSG2UHEFD6AHTZHSTNSYMTCPYHQ6EBCF3U7577IKKASHT3QNQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.611966581 (7h29m55.199022822s) [noise=true , address=104.36.82.46:4509 , piece_id=7WMUNECBF2QDOJCSONQGZZ5OTIY7P6QWYAOHOQKJFZS77ADXZPJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.627727450 (7h29m55.214783701s) [noise=true , address=104.36.82.47:4539 , piece_id=WLHS5IUZ5ZFLMGZSMRHSH3XBJKN3W4XKSJYDE5JZFUYHNQU5PDQQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.645759177 (7h29m55.232815418s) [noise=true , address=104.36.82.37:4666 , piece_id=V2EI6I3LJ7ZMGWDVQQRBZSP2JUWI7CVIEUG5PWH53AGKL3P5BEPA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.774571312 (7h29m55.361627554s) [noise=true , address=104.36.82.13:4193 , piece_id=YR3ACGAGY2A5S5I75H2KCGU3ENI2FN3UCPDRXFP4QL4TAEEX5X6Q, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:41.876799657 (7h29m55.463855878s) [noise=true , address=104.36.82.8:4999 , piece_id=TSRYX3J6AO3XEWP3KNYCYUGYQ3RRWFPSHZPPPL2ZSQUKPAS5HHTQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000385207 (7h29m55.587441438s) [noise=true , address=64.49.13.139:4470 , piece_id=OQO6YNTSUHDBYSQ7B5QAZ3QCJSV3MW3XPFEBXQ2LW764HYBXLDTA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: <nil> Aug 7 23:56:42.000439059 (7h29m55.587495289s) [seg_pos={0 0}, upload=7685]: Segment reached optimal threshold of 14 pieces. Aug 7 23:56:42.000640734 (7h29m55.587696955s) [noise=true , address=64.49.13.170:4717 , piece_id=TKUOEBYDXKVK4ZSMDXLSQ6ZETIBYYEJR3I3EKNRVGY62MXO5BEJA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000697911 (7h29m55.587754142s) [noise=true , address=64.49.13.137:4495 , piece_id=AZKX4MZQJ4JGVPRZBLDTOEV5VYP4NYPQIT3WOSW4UOD3VF5G2RWA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dgidHuoyZFQZQyBAfQrBAn3FXjgePo8GjuzPyBijKp5xb4wk1): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000755489 (7h29m55.58781173s) [noise=true , address=64.49.13.170:4718 , piece_id=RDBMQ763YESCMLPW6744NA7CZKOTBPHX6XFYTHW2O7DW6XDXJ2ZQ, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AcLsSmVz9sRtuDLs47WoWYC66YSbr2aXBjzBdjayg22daQ2Hp): protocol: expected piece hash; context canceled; EOF Aug 7 23:56:42.000812144 (7h29m55.587868355s) [noise=true , address=104.36.82.5:4851 , piece_id=ZMGB2HAZBF3NICA4FRS75BC5KFQB3TOC6QXNPC52NMYB4QXBTSYA, seg_pos={0 0}, upload=7685]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fTtTieEUquaKNUNzqK98PkX7tVxjsiNeMLjz8kQJwdLne7d9R): protocol: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15446->104.36.82.5:4851: use of closed network connection Aug 7 23:56:42.000840427 (7h29m55.587896659s) [seg_pos={0 0}, upload=7685]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:56:42.000912191 (7h29m55.587968402s) [upload=7685]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:56:42.000916679 (7h29m55.5879729s) [upload=7685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:01:47.368617454 (7h35m0.955673666s) [upload=7685]: Done uploading segments... Aug 8 00:01:47.368640257 (7h35m0.955696468s) [upload=7685]: Upload finished. err: metaclient: context deadline exceeded Duration: 5m21.270882235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 4 out of 60 is failed (threshold=3) [XX..X....................................................X..] duration: 241.809129ms FAIL (2025-08-08 00:01:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 241.832122ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 5 out of 60 is failed (threshold=3) [X.X...............................................XX.X......] duration: 222.7728ms FAIL (2025-08-08 00:01:34 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Duration:* 222.796614ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 163.392979ms FAIL (2025-08-08 00:01:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 163.418116ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 11 out of 60 is failed (threshold=3) [X............................................XXXXXXXXX...X..] duration: 193.517424ms FAIL (2025-08-08 00:01:27 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 193.532532ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 179.630564ms FAIL (2025-08-08 00:01:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 179.653488ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X.X...........................................X.X..XXX....] duration: 2.006631469s FAIL (2025-08-08 00:01:11 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.006658117s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.657465742s FAIL (2025-08-08 00:01:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 2.65749117s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X....XXX.X......................................XXXXX.....] duration: 418.866433ms FAIL (2025-08-08 00:01:04 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== , seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522017724 (7h27m3.109073975s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522360804 (7h27m3.109417046s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522463095 (7h27m3.109519336s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522783393 (7h27m3.109839644s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522884061 (7h27m3.109940302s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523171748 (7h27m3.110228s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523292534 (7h27m3.110348775s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523596000 (7h27m3.110652251s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523713870 (7h27m3.110770111s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524251423 (7h27m3.111307674s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524718816 (7h27m3.111775057s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.647138359 (7h27m3.2341946s) [upload=7640]: Next returned nil segment Aug 7 23:53:49.647155511 (7h27m3.234211752s) [upload=7640]: Waiting for error group managing segments... Aug 7 23:53:49.655170752 (7h27m3.242226993s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.655792202 (7h27m3.242848433s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656163956 (7h27m3.243220187s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656798870 (7h27m3.243855091s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.657996216 (7h27m3.245052458s) [noise=true , address=45.91.116.4:4939 , piece_id=LWLSQN6BSOJY4PA7I7HZL5PSRI4DNR6E4GMYWPUB6DF6BLMPU26A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.658644787 (7h27m3.245701017s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.659153586 (7h27m3.246209817s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660604835 (7h27m3.247661076s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660941473 (7h27m3.247997694s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.661457717 (7h27m3.248513959s) [noise=true , address=77.90.31.18:4807 , piece_id=5CCUKS55ROKTHMZA7NKUNYXOTAKYZIDQGDJBSMFM5TSBEFVDJMPA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.662043810 (7h27m3.249100041s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665007553 (7h27m3.252063794s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665952497 (7h27m3.253008727s) [noise=true , address=77.90.31.16:4734 , piece_id=AIAJVZAYQTA4OR6WDJ6V2XH3TXBO6FA4VRUCYJGIFXXUO3ZGIT4A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665980449 (7h27m3.25303667s) [seg_pos={0 0}, upload=7640]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:49.666316626 (7h27m3.253372857s) [noise=true , address=104.36.82.14:4221 , piece_id=JKDWKECRRZCOLROOZ5QXNOGIRN3LLLAUGDQRFBPDYBHBV6LTPH7Q, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: rpc: dial tcp 104.36.82.14:4221: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666330082 (7h27m3.253386322s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: read tcp 5.161.89.252:14750->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666492835 (7h27m3.253549076s) [noise=true , address=104.36.82.16:4294 , piece_id=PL7A2YIDU7ZU2F5J4IE37IFPYADS2VNPLY443J46RIBG7J6ZNZJA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: read tcp 5.161.89.252:34080->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666519565 (7h27m3.253575806s) [seg_pos={0 0}, upload=7640]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:49.666614813 (7h27m3.253671053s) [upload=7640]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:49.666620242 (7h27m3.253676484s) [upload=7640]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:17.275326827 (7h30m30.862383048s) [upload=7640]: Done uploading segments... Aug 7 23:57:17.275353585 (7h30m30.862409807s) [upload=7640]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:17.349138524 (7h30m30.936194735s) [upload=7698]: Starting upload Aug 7 23:57:17.349187294 (7h30m30.936243515s) [upload=7698]: Done starting upload Aug 7 23:57:17.349221879 (7h30m30.93627809s) [upload=7698]: Uploading segments... Aug 7 23:57:17.349404010 (7h30m30.93646022s) [upload=7698]: Got next segment. Inline: false Aug 7 23:57:17.349444765 (7h30m30.936500976s) [upload=7698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:17.349448802 (7h30m30.936505023s) [upload=7698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:47.334809460 (7h31m0.921865702s) [upload=7698]: Next segment err: context deadline exceeded Aug 7 23:57:47.334985600 (7h31m0.922041831s) [upload=7698]: Done uploading segments... Aug 7 23:57:47.335006539 (7h31m0.922062781s) [upload=7698]: Upload finished. err: context deadline exceeded Aug 7 23:57:47.412546828 (7h31m0.99960307s) [upload=7705]: Starting upload Aug 7 23:57:47.412626116 (7h31m0.999682357s) [upload=7705]: Done starting upload Aug 7 23:57:47.412754536 (7h31m0.999810777s) [upload=7705]: Uploading segments... Aug 7 23:57:47.413018508 (7h31m1.000074749s) [upload=7705]: Got next segment. Inline: false Aug 7 23:57:47.413067800 (7h31m1.000124042s) [upload=7705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:47.413073351 (7h31m1.000129592s) [upload=7705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:17.397856277 (7h31m30.984912519s) [upload=7705]: Next segment err: context deadline exceeded Aug 7 23:58:17.397964409 (7h31m30.98502065s) [upload=7705]: Done uploading segments... Aug 7 23:58:17.397983265 (7h31m30.985039505s) [upload=7705]: Upload finished. err: context deadline exceeded Aug 7 23:58:17.497021948 (7h31m31.084078188s) [upload=7712]: Starting upload Aug 7 23:58:17.497166287 (7h31m31.084222528s) [upload=7712]: Done starting upload Aug 7 23:58:17.497376509 (7h31m31.084432751s) [upload=7712]: Uploading segments... Aug 7 23:58:17.497668814 (7h31m31.084725066s) [upload=7712]: Got next segment. Inline: false Aug 7 23:58:17.497771346 (7h31m31.084827597s) [upload=7712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:17.497783308 (7h31m31.08483955s) [upload=7712]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:47.474157820 (7h32m1.061214041s) [upload=7712]: Next segment err: context deadline exceeded Aug 7 23:58:47.474221107 (7h32m1.061277319s) [upload=7712]: Done uploading segments... Aug 7 23:58:47.474236366 (7h32m1.061292587s) [upload=7712]: Upload finished. err: context deadline exceeded Duration: 418.883756ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [XXX..X....................................................X.] duration: 248.296009ms FAIL (2025-08-08 00:01:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 248.315506ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.X..........................................X...........] duration: 527.070833ms FAIL (2025-08-08 00:00:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 527.091081ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [.X..X..X......................................XXX.X.XXX.X..X] duration: 30.001095253s FAIL (2025-08-08 00:00:49 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001113578s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X....XXX.X......................................XXXXX....] duration: 651.700041ms FAIL (2025-08-08 00:00:48 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== , seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522017724 (7h27m3.109073975s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522360804 (7h27m3.109417046s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522463095 (7h27m3.109519336s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522783393 (7h27m3.109839644s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522884061 (7h27m3.109940302s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523171748 (7h27m3.110228s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523292534 (7h27m3.110348775s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523596000 (7h27m3.110652251s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523713870 (7h27m3.110770111s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524251423 (7h27m3.111307674s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524718816 (7h27m3.111775057s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.647138359 (7h27m3.2341946s) [upload=7640]: Next returned nil segment Aug 7 23:53:49.647155511 (7h27m3.234211752s) [upload=7640]: Waiting for error group managing segments... Aug 7 23:53:49.655170752 (7h27m3.242226993s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.655792202 (7h27m3.242848433s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656163956 (7h27m3.243220187s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656798870 (7h27m3.243855091s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.657996216 (7h27m3.245052458s) [noise=true , address=45.91.116.4:4939 , piece_id=LWLSQN6BSOJY4PA7I7HZL5PSRI4DNR6E4GMYWPUB6DF6BLMPU26A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.658644787 (7h27m3.245701017s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.659153586 (7h27m3.246209817s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660604835 (7h27m3.247661076s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660941473 (7h27m3.247997694s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.661457717 (7h27m3.248513959s) [noise=true , address=77.90.31.18:4807 , piece_id=5CCUKS55ROKTHMZA7NKUNYXOTAKYZIDQGDJBSMFM5TSBEFVDJMPA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.662043810 (7h27m3.249100041s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665007553 (7h27m3.252063794s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665952497 (7h27m3.253008727s) [noise=true , address=77.90.31.16:4734 , piece_id=AIAJVZAYQTA4OR6WDJ6V2XH3TXBO6FA4VRUCYJGIFXXUO3ZGIT4A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665980449 (7h27m3.25303667s) [seg_pos={0 0}, upload=7640]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:49.666316626 (7h27m3.253372857s) [noise=true , address=104.36.82.14:4221 , piece_id=JKDWKECRRZCOLROOZ5QXNOGIRN3LLLAUGDQRFBPDYBHBV6LTPH7Q, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: rpc: dial tcp 104.36.82.14:4221: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666330082 (7h27m3.253386322s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: read tcp 5.161.89.252:14750->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666492835 (7h27m3.253549076s) [noise=true , address=104.36.82.16:4294 , piece_id=PL7A2YIDU7ZU2F5J4IE37IFPYADS2VNPLY443J46RIBG7J6ZNZJA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: read tcp 5.161.89.252:34080->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666519565 (7h27m3.253575806s) [seg_pos={0 0}, upload=7640]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:49.666614813 (7h27m3.253671053s) [upload=7640]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:49.666620242 (7h27m3.253676484s) [upload=7640]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:17.275326827 (7h30m30.862383048s) [upload=7640]: Done uploading segments... Aug 7 23:57:17.275353585 (7h30m30.862409807s) [upload=7640]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:17.349138524 (7h30m30.936194735s) [upload=7698]: Starting upload Aug 7 23:57:17.349187294 (7h30m30.936243515s) [upload=7698]: Done starting upload Aug 7 23:57:17.349221879 (7h30m30.93627809s) [upload=7698]: Uploading segments... Aug 7 23:57:17.349404010 (7h30m30.93646022s) [upload=7698]: Got next segment. Inline: false Aug 7 23:57:17.349444765 (7h30m30.936500976s) [upload=7698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:17.349448802 (7h30m30.936505023s) [upload=7698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:47.334809460 (7h31m0.921865702s) [upload=7698]: Next segment err: context deadline exceeded Aug 7 23:57:47.334985600 (7h31m0.922041831s) [upload=7698]: Done uploading segments... Aug 7 23:57:47.335006539 (7h31m0.922062781s) [upload=7698]: Upload finished. err: context deadline exceeded Aug 7 23:57:47.412546828 (7h31m0.99960307s) [upload=7705]: Starting upload Aug 7 23:57:47.412626116 (7h31m0.999682357s) [upload=7705]: Done starting upload Aug 7 23:57:47.412754536 (7h31m0.999810777s) [upload=7705]: Uploading segments... Aug 7 23:57:47.413018508 (7h31m1.000074749s) [upload=7705]: Got next segment. Inline: false Aug 7 23:57:47.413067800 (7h31m1.000124042s) [upload=7705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:47.413073351 (7h31m1.000129592s) [upload=7705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:17.397856277 (7h31m30.984912519s) [upload=7705]: Next segment err: context deadline exceeded Aug 7 23:58:17.397964409 (7h31m30.98502065s) [upload=7705]: Done uploading segments... Aug 7 23:58:17.397983265 (7h31m30.985039505s) [upload=7705]: Upload finished. err: context deadline exceeded Aug 7 23:58:17.497021948 (7h31m31.084078188s) [upload=7712]: Starting upload Aug 7 23:58:17.497166287 (7h31m31.084222528s) [upload=7712]: Done starting upload Aug 7 23:58:17.497376509 (7h31m31.084432751s) [upload=7712]: Uploading segments... Aug 7 23:58:17.497668814 (7h31m31.084725066s) [upload=7712]: Got next segment. Inline: false Aug 7 23:58:17.497771346 (7h31m31.084827597s) [upload=7712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:17.497783308 (7h31m31.08483955s) [upload=7712]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:47.474157820 (7h32m1.061214041s) [upload=7712]: Next segment err: context deadline exceeded Aug 7 23:58:47.474221107 (7h32m1.061277319s) [upload=7712]: Done uploading segments... Aug 7 23:58:47.474236366 (7h32m1.061292587s) [upload=7712]: Upload finished. err: context deadline exceeded Duration: 651.728415ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [XXXX..X....................................................X] duration: 30.001315188s FAIL (2025-08-08 00:00:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00133245s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 6 out of 60 is failed (threshold=3) [XX.X...............................................XX.X.....] duration: 206.286189ms FAIL (2025-08-08 00:00:25 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Duration:* 206.302969ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 235.656187ms FAIL (2025-08-08 00:00:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ne Aug 7 23:09:39.944386324 (6h42m53.531442545s) [upload=6887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:09:40.134449947 (6h42m53.721506158s) [upload=6887]: Done uploading segments... Aug 7 23:09:40.134464625 (6h42m53.721520846s) [upload=6887]: Upload finished. err: <nil> Aug 7 23:10:38.125235482 (6h43m51.712291723s) [upload=6908]: Starting upload Aug 7 23:10:38.125321953 (6h43m51.712378205s) [upload=6908]: Done starting upload Aug 7 23:10:38.125506868 (6h43m51.712563119s) [upload=6908]: Uploading segments... Aug 7 23:10:38.125577140 (6h43m51.712633391s) [upload=6908]: Got next segment. Inline: true Aug 7 23:10:38.125584223 (6h43m51.712640464s) [upload=6908]: Waiting for error group managing segments... Aug 7 23:10:38.125639727 (6h43m51.712695977s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:10:38.125651738 (6h43m51.71270799s) [upload=6908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:10:38.125657910 (6h43m51.712714161s) [upload=6908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:10:38.249068118 (6h43m51.836124368s) [upload=6908]: Done uploading segments... Aug 7 23:10:38.249095648 (6h43m51.836151889s) [upload=6908]: Upload finished. err: <nil> Aug 7 23:11:37.996040842 (6h44m51.583097123s) [upload=6923]: Starting upload Aug 7 23:11:37.996093240 (6h44m51.583149461s) [upload=6923]: Done starting upload Aug 7 23:11:37.996137332 (6h44m51.583193553s) [upload=6923]: Uploading segments... Aug 7 23:11:37.996236618 (6h44m51.58329286s) [upload=6923]: Got next segment. Inline: true Aug 7 23:11:37.996246336 (6h44m51.583302557s) [upload=6923]: Waiting for error group managing segments... Aug 7 23:11:37.996301449 (6h44m51.58335767s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:11:37.996308312 (6h44m51.583364553s) [upload=6923]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:11:37.996313571 (6h44m51.583369783s) [upload=6923]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:11:38.115493220 (6h44m51.702549431s) [upload=6923]: Done uploading segments... Aug 7 23:11:38.115513639 (6h44m51.702569849s) [upload=6923]: Upload finished. err: <nil> Aug 7 23:12:37.449808013 (6h45m51.036864224s) [upload=6941]: Starting upload Aug 7 23:12:37.449858507 (6h45m51.036914718s) [upload=6941]: Done starting upload Aug 7 23:12:37.449904994 (6h45m51.036961205s) [upload=6941]: Uploading segments... Aug 7 23:12:37.449949496 (6h45m51.037005707s) [upload=6941]: Got next segment. Inline: true Aug 7 23:12:37.449955127 (6h45m51.037011338s) [upload=6941]: Waiting for error group managing segments... Aug 7 23:12:37.449995823 (6h45m51.037052034s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:12:37.450001143 (6h45m51.037057354s) [upload=6941]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:12:37.450004189 (6h45m51.0370604s) [upload=6941]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:12:37.534556583 (6h45m51.121612794s) [upload=6941]: Done uploading segments... Aug 7 23:12:37.534575238 (6h45m51.121631449s) [upload=6941]: Upload finished. err: <nil> Aug 7 23:13:12.303607944 (6h46m25.890664155s) [upload=6950]: Starting upload Aug 7 23:13:12.303650153 (6h46m25.890706374s) [upload=6950]: Done starting upload Aug 7 23:13:12.303719743 (6h46m25.890775964s) [upload=6950]: Uploading segments... Aug 7 23:13:12.303750360 (6h46m25.890806581s) [upload=6950]: Got next segment. Inline: true Aug 7 23:13:12.303754137 (6h46m25.890810358s) [upload=6950]: Waiting for error group managing segments... Aug 7 23:13:12.303786066 (6h46m25.890842278s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:12.303789663 (6h46m25.890845884s) [upload=6950]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:12.303792177 (6h46m25.890848398s) [upload=6950]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:12.374674527 (6h46m25.961730738s) [upload=6950]: Done uploading segments... Aug 7 23:13:12.374688663 (6h46m25.961744874s) [upload=6950]: Upload finished. err: <nil> Aug 7 23:13:56.776579420 (6h47m10.36363566s) [upload=6962]: Starting upload Aug 7 23:13:56.776644211 (6h47m10.363700462s) [upload=6962]: Done starting upload Aug 7 23:13:56.776723829 (6h47m10.363780071s) [upload=6962]: Uploading segments... Aug 7 23:13:56.776773401 (6h47m10.363829643s) [upload=6962]: Got next segment. Inline: true Aug 7 23:13:56.776779252 (6h47m10.363835494s) [upload=6962]: Waiting for error group managing segments... Aug 7 23:13:56.776832092 (6h47m10.363888332s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:13:56.776839315 (6h47m10.363895556s) [upload=6962]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:13:56.776843042 (6h47m10.363899293s) [upload=6962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:13:56.833253067 (6h47m10.420309309s) [upload=6962]: Done uploading segments... Aug 7 23:13:56.833272594 (6h47m10.420328835s) [upload=6962]: Upload finished. err: <nil> Aug 7 23:14:23.847990399 (6h47m37.43504664s) [upload=6971]: Starting upload Aug 7 23:14:23.848064437 (6h47m37.435120678s) [upload=6971]: Done starting upload Aug 7 23:14:23.848133346 (6h47m37.435189557s) [upload=6971]: Uploading segments... Aug 7 23:14:23.848226790 (6h47m37.435283002s) [upload=6971]: Got next segment. Inline: true Aug 7 23:14:23.848231990 (6h47m37.435288201s) [upload=6971]: Waiting for error group managing segments... Aug 7 23:14:23.848270613 (6h47m37.435326823s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:23.848275903 (6h47m37.435332124s) [upload=6971]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:23.848278567 (6h47m37.435334779s) [upload=6971]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:24.081872322 (6h47m37.668928543s) [upload=6971]: Done uploading segments... Aug 7 23:14:24.081896036 (6h47m37.668952257s) [upload=6971]: Upload finished. err: <nil> Aug 7 23:14:49.940481761 (6h48m3.527538001s) [upload=6979]: Starting upload Aug 7 23:14:49.940552292 (6h48m3.527608533s) [upload=6979]: Done starting upload Aug 7 23:14:49.940622082 (6h48m3.527678324s) [upload=6979]: Uploading segments... Aug 7 23:14:49.940755703 (6h48m3.527811943s) [upload=6979]: Got next segment. Inline: true Aug 7 23:14:49.940763758 (6h48m3.527819998s) [upload=6979]: Waiting for error group managing segments... Aug 7 23:14:49.940819711 (6h48m3.527875952s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:14:49.940826003 (6h48m3.527882244s) [upload=6979]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:14:49.940829760 (6h48m3.527886001s) [upload=6979]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:14:50.031323214 (6h48m3.618379435s) [upload=6979]: Done uploading segments... Aug 7 23:14:50.031337651 (6h48m3.618393872s) [upload=6979]: Upload finished. err: <nil> Aug 7 23:15:44.466066460 (6h48m58.053122681s) [upload=6995]: Starting upload Aug 7 23:15:44.466127083 (6h48m58.053183304s) [upload=6995]: Done starting upload Aug 7 23:15:44.466180152 (6h48m58.053236374s) [upload=6995]: Uploading segments... Aug 7 23:15:44.466238350 (6h48m58.053294571s) [upload=6995]: Got next segment. Inline: true Aug 7 23:15:44.466243571 (6h48m58.053299781s) [upload=6995]: Waiting for error group managing segments... Aug 7 23:15:44.466279948 (6h48m58.05333617s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:15:44.466284317 (6h48m58.053340538s) [upload=6995]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:15:44.466286601 (6h48m58.053342812s) [upload=6995]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:15:44.537462409 (6h48m58.124518621s) [upload=6995]: Done uploading segments... Aug 7 23:15:44.537479490 (6h48m58.124535701s) [upload=6995]: Upload finished. err: <nil> Aug 7 23:16:06.153565692 (6h49m19.740621913s) [upload=6998]: Starting upload Aug 7 23:16:06.153632095 (6h49m19.740688306s) [upload=6998]: Done starting upload Aug 7 23:16:06.153689062 (6h49m19.740745293s) [upload=6998]: Uploading segments... Aug 7 23:16:06.153727503 (6h49m19.740783714s) [upload=6998]: Got next segment. Inline: true Aug 7 23:16:06.153733284 (6h49m19.740789505s) [upload=6998]: Waiting for error group managing segments... Aug 7 23:16:06.153775644 (6h49m19.740831865s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:06.153781144 (6h49m19.740837365s) [upload=6998]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:06.153785161 (6h49m19.740841372s) [upload=6998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:06.348027973 (6h49m19.935084194s) [upload=6998]: Done uploading segments... Aug 7 23:16:06.348046217 (6h49m19.935102438s) [upload=6998]: Upload finished. err: <nil> Aug 7 23:16:30.802675866 (6h49m44.389732087s) [upload=7006]: Starting upload Aug 7 23:16:30.802719959 (6h49m44.38977618s) [upload=7006]: Done starting upload Aug 7 23:16:30.802764312 (6h49m44.389820533s) [upload=7006]: Uploading segments... Aug 7 23:16:30.802792133 (6h49m44.389848344s) [upload=7006]: Got next segment. Inline: true Aug 7 23:16:30.802795820 (6h49m44.389852031s) [upload=7006]: Waiting for error group managing segments... Aug 7 23:16:30.802827569 (6h49m44.38988378s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:16:30.802831226 (6h49m44.389887437s) [upload=7006]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:16:30.802833681 (6h49m44.389889902s) [upload=7006]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:16:30.805506480 (6h49m44.392562691s) [upload=7006]: Done uploading segments... Aug 7 23:16:30.805550723 (6h49m44.392606934s) [upload=7006]: Upload finished. err: metaclient: Too Many Requests Duration:* 235.683929ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [XXXX..X............................................XXXXXXX..] duration: 4.645024316s FAIL (2025-08-08 00:00:23 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.64504265s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X....XXX.X......................................XXXXX...] duration: 430.798714ms FAIL (2025-08-08 00:00:21 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== , seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522017724 (7h27m3.109073975s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522360804 (7h27m3.109417046s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522463095 (7h27m3.109519336s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522783393 (7h27m3.109839644s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522884061 (7h27m3.109940302s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523171748 (7h27m3.110228s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523292534 (7h27m3.110348775s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523596000 (7h27m3.110652251s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523713870 (7h27m3.110770111s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524251423 (7h27m3.111307674s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524718816 (7h27m3.111775057s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.647138359 (7h27m3.2341946s) [upload=7640]: Next returned nil segment Aug 7 23:53:49.647155511 (7h27m3.234211752s) [upload=7640]: Waiting for error group managing segments... Aug 7 23:53:49.655170752 (7h27m3.242226993s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.655792202 (7h27m3.242848433s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656163956 (7h27m3.243220187s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656798870 (7h27m3.243855091s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.657996216 (7h27m3.245052458s) [noise=true , address=45.91.116.4:4939 , piece_id=LWLSQN6BSOJY4PA7I7HZL5PSRI4DNR6E4GMYWPUB6DF6BLMPU26A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.658644787 (7h27m3.245701017s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.659153586 (7h27m3.246209817s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660604835 (7h27m3.247661076s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660941473 (7h27m3.247997694s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.661457717 (7h27m3.248513959s) [noise=true , address=77.90.31.18:4807 , piece_id=5CCUKS55ROKTHMZA7NKUNYXOTAKYZIDQGDJBSMFM5TSBEFVDJMPA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.662043810 (7h27m3.249100041s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665007553 (7h27m3.252063794s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665952497 (7h27m3.253008727s) [noise=true , address=77.90.31.16:4734 , piece_id=AIAJVZAYQTA4OR6WDJ6V2XH3TXBO6FA4VRUCYJGIFXXUO3ZGIT4A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665980449 (7h27m3.25303667s) [seg_pos={0 0}, upload=7640]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:49.666316626 (7h27m3.253372857s) [noise=true , address=104.36.82.14:4221 , piece_id=JKDWKECRRZCOLROOZ5QXNOGIRN3LLLAUGDQRFBPDYBHBV6LTPH7Q, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: rpc: dial tcp 104.36.82.14:4221: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666330082 (7h27m3.253386322s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: read tcp 5.161.89.252:14750->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666492835 (7h27m3.253549076s) [noise=true , address=104.36.82.16:4294 , piece_id=PL7A2YIDU7ZU2F5J4IE37IFPYADS2VNPLY443J46RIBG7J6ZNZJA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: read tcp 5.161.89.252:34080->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666519565 (7h27m3.253575806s) [seg_pos={0 0}, upload=7640]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:49.666614813 (7h27m3.253671053s) [upload=7640]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:49.666620242 (7h27m3.253676484s) [upload=7640]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:17.275326827 (7h30m30.862383048s) [upload=7640]: Done uploading segments... Aug 7 23:57:17.275353585 (7h30m30.862409807s) [upload=7640]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:17.349138524 (7h30m30.936194735s) [upload=7698]: Starting upload Aug 7 23:57:17.349187294 (7h30m30.936243515s) [upload=7698]: Done starting upload Aug 7 23:57:17.349221879 (7h30m30.93627809s) [upload=7698]: Uploading segments... Aug 7 23:57:17.349404010 (7h30m30.93646022s) [upload=7698]: Got next segment. Inline: false Aug 7 23:57:17.349444765 (7h30m30.936500976s) [upload=7698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:17.349448802 (7h30m30.936505023s) [upload=7698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:47.334809460 (7h31m0.921865702s) [upload=7698]: Next segment err: context deadline exceeded Aug 7 23:57:47.334985600 (7h31m0.922041831s) [upload=7698]: Done uploading segments... Aug 7 23:57:47.335006539 (7h31m0.922062781s) [upload=7698]: Upload finished. err: context deadline exceeded Aug 7 23:57:47.412546828 (7h31m0.99960307s) [upload=7705]: Starting upload Aug 7 23:57:47.412626116 (7h31m0.999682357s) [upload=7705]: Done starting upload Aug 7 23:57:47.412754536 (7h31m0.999810777s) [upload=7705]: Uploading segments... Aug 7 23:57:47.413018508 (7h31m1.000074749s) [upload=7705]: Got next segment. Inline: false Aug 7 23:57:47.413067800 (7h31m1.000124042s) [upload=7705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:47.413073351 (7h31m1.000129592s) [upload=7705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:17.397856277 (7h31m30.984912519s) [upload=7705]: Next segment err: context deadline exceeded Aug 7 23:58:17.397964409 (7h31m30.98502065s) [upload=7705]: Done uploading segments... Aug 7 23:58:17.397983265 (7h31m30.985039505s) [upload=7705]: Upload finished. err: context deadline exceeded Aug 7 23:58:17.497021948 (7h31m31.084078188s) [upload=7712]: Starting upload Aug 7 23:58:17.497166287 (7h31m31.084222528s) [upload=7712]: Done starting upload Aug 7 23:58:17.497376509 (7h31m31.084432751s) [upload=7712]: Uploading segments... Aug 7 23:58:17.497668814 (7h31m31.084725066s) [upload=7712]: Got next segment. Inline: false Aug 7 23:58:17.497771346 (7h31m31.084827597s) [upload=7712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:17.497783308 (7h31m31.08483955s) [upload=7712]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:47.474157820 (7h32m1.061214041s) [upload=7712]: Next segment err: context deadline exceeded Aug 7 23:58:47.474221107 (7h32m1.061277319s) [upload=7712]: Done uploading segments... Aug 7 23:58:47.474236366 (7h32m1.061292587s) [upload=7712]: Upload finished. err: context deadline exceeded Duration: 430.825824ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 12 out of 60 is failed (threshold=3) [XX............................................XXXXXXXXX...X.] duration: 258.341087ms FAIL (2025-08-08 00:00:20 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 (7h26m36.719617128s) [upload=7626]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:53:28.217132518 (7h26m41.804188759s) [upload=7638]: Starting upload Aug 7 23:53:28.217216185 (7h26m41.804272426s) [upload=7638]: Done starting upload Aug 7 23:53:28.217288009 (7h26m41.80434426s) [upload=7638]: Uploading segments... Aug 7 23:53:28.217336129 (7h26m41.80439237s) [upload=7638]: Got next segment. Inline: true Aug 7 23:53:28.217341890 (7h26m41.804398131s) [upload=7638]: Waiting for error group managing segments... Aug 7 23:53:28.217421438 (7h26m41.804477679s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:28.217427650 (7h26m41.804483901s) [upload=7638]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:28.217431617 (7h26m41.804487859s) [upload=7638]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:53:58.204830989 (7h27m11.79188721s) [upload=7638]: Done uploading segments... Aug 7 23:53:58.204849294 (7h27m11.791905505s) [upload=7638]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:01.364416813 (7h27m14.951473054s) [upload=7648]: Starting upload Aug 7 23:54:01.364544481 (7h27m14.951600732s) [upload=7648]: Done starting upload Aug 7 23:54:01.364639518 (7h27m14.95169577s) [upload=7648]: Uploading segments... Aug 7 23:54:01.364703858 (7h27m14.951760109s) [upload=7648]: Got next segment. Inline: true Aug 7 23:54:01.364713517 (7h27m14.951769757s) [upload=7648]: Waiting for error group managing segments... Aug 7 23:54:01.364781374 (7h27m14.951837625s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:01.364791231 (7h27m14.951847472s) [upload=7648]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:01.364797502 (7h27m14.951853754s) [upload=7648]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:31.351186626 (7h27m44.938242847s) [upload=7648]: Done uploading segments... Aug 7 23:54:31.351241118 (7h27m44.938297339s) [upload=7648]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:31.416140839 (7h27m45.003197049s) [upload=7656]: Starting upload Aug 7 23:54:31.416189309 (7h27m45.00324553s) [upload=7656]: Done starting upload Aug 7 23:54:31.416235946 (7h27m45.003292157s) [upload=7656]: Uploading segments... Aug 7 23:54:31.416277384 (7h27m45.003333595s) [upload=7656]: Got next segment. Inline: true Aug 7 23:54:31.416297601 (7h27m45.003353813s) [upload=7656]: Waiting for error group managing segments... Aug 7 23:54:31.416336454 (7h27m45.003392665s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:31.416340652 (7h27m45.003396873s) [upload=7656]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:31.416342965 (7h27m45.003399177s) [upload=7656]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:01.405081824 (7h28m14.992138035s) [upload=7656]: Done uploading segments... Aug 7 23:55:01.405101451 (7h28m14.992157672s) [upload=7656]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:45.764200449 (7h28m59.35125669s) [upload=7673]: Starting upload Aug 7 23:55:45.764300708 (7h28m59.351356948s) [upload=7673]: Done starting upload Aug 7 23:55:45.764398370 (7h28m59.351454611s) [upload=7673]: Uploading segments... Aug 7 23:55:45.764515538 (7h28m59.35157178s) [upload=7673]: Got next segment. Inline: true Aug 7 23:55:45.764521980 (7h28m59.351578222s) [upload=7673]: Waiting for error group managing segments... Aug 7 23:55:45.764570060 (7h28m59.3516263s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:45.764575871 (7h28m59.351632111s) [upload=7673]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:45.764579898 (7h28m59.351636149s) [upload=7673]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:15.747212127 (7h29m29.334268369s) [upload=7673]: Done uploading segments... Aug 7 23:56:15.747307315 (7h29m29.334363566s) [upload=7673]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:15.823083783 (7h29m29.410140035s) [upload=7682]: Starting upload Aug 7 23:56:15.823169003 (7h29m29.410225243s) [upload=7682]: Done starting upload Aug 7 23:56:15.823245125 (7h29m29.410301367s) [upload=7682]: Uploading segments... Aug 7 23:56:15.823289758 (7h29m29.410346009s) [upload=7682]: Got next segment. Inline: true Aug 7 23:56:15.823295359 (7h29m29.410351599s) [upload=7682]: Waiting for error group managing segments... Aug 7 23:56:15.823343018 (7h29m29.410399269s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:15.823348678 (7h29m29.41040492s) [upload=7682]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:15.823352376 (7h29m29.410408616s) [upload=7682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:45.810149218 (7h29m59.397205439s) [upload=7682]: Done uploading segments... Aug 7 23:56:45.810167081 (7h29m59.397223322s) [upload=7682]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:56:58.169907263 (7h30m11.756963514s) [upload=7692]: Starting upload Aug 7 23:56:58.169972405 (7h30m11.757028647s) [upload=7692]: Done starting upload Aug 7 23:56:58.170060319 (7h30m11.75711657s) [upload=7692]: Uploading segments... Aug 7 23:56:58.170104551 (7h30m11.757160792s) [upload=7692]: Got next segment. Inline: true Aug 7 23:56:58.170110071 (7h30m11.757166313s) [upload=7692]: Waiting for error group managing segments... Aug 7 23:56:58.170155567 (7h30m11.757211808s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:56:58.170161878 (7h30m11.75721812s) [upload=7692]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:56:58.170165555 (7h30m11.757221807s) [upload=7692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:28.152393221 (7h30m41.739449472s) [upload=7692]: Done uploading segments... Aug 7 23:57:28.152443946 (7h30m41.739500187s) [upload=7692]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:58:12.872539723 (7h31m26.459595944s) [upload=7710]: Starting upload Aug 7 23:58:12.872620014 (7h31m26.459676225s) [upload=7710]: Done starting upload Aug 7 23:58:12.872680977 (7h31m26.459737199s) [upload=7710]: Uploading segments... Aug 7 23:58:12.872720942 (7h31m26.459777153s) [upload=7710]: Got next segment. Inline: true Aug 7 23:58:12.872726774 (7h31m26.459783014s) [upload=7710]: Waiting for error group managing segments... Aug 7 23:58:12.872773470 (7h31m26.459829691s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:12.872779180 (7h31m26.459835401s) [upload=7710]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:12.872783188 (7h31m26.459839409s) [upload=7710]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:12.946938777 (7h31m26.533994998s) [upload=7710]: Done uploading segments... Aug 7 23:58:12.946953044 (7h31m26.534009265s) [upload=7710]: Upload finished. err: <nil> Aug 7 23:58:33.964575085 (7h31m47.551631306s) [upload=7716]: Starting upload Aug 7 23:58:33.964639436 (7h31m47.551695646s) [upload=7716]: Done starting upload Aug 7 23:58:33.964700178 (7h31m47.551756429s) [upload=7716]: Uploading segments... Aug 7 23:58:33.964768666 (7h31m47.551824907s) [upload=7716]: Got next segment. Inline: true Aug 7 23:58:33.964776340 (7h31m47.551832581s) [upload=7716]: Waiting for error group managing segments... Aug 7 23:58:33.964826585 (7h31m47.551882826s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:33.964832877 (7h31m47.551889117s) [upload=7716]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:33.964837114 (7h31m47.551893355s) [upload=7716]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:34.023384420 (7h31m47.610440681s) [upload=7716]: Done uploading segments... Aug 7 23:58:34.023410589 (7h31m47.610466841s) [upload=7716]: Upload finished. err: <nil> Aug 7 23:58:59.700940485 (7h32m13.287996695s) [upload=7725]: Starting upload Aug 7 23:58:59.701013871 (7h32m13.288070092s) [upload=7725]: Done starting upload Aug 7 23:58:59.701058494 (7h32m13.288114715s) [upload=7725]: Uploading segments... Aug 7 23:58:59.701105913 (7h32m13.288162144s) [upload=7725]: Got next segment. Inline: true Aug 7 23:58:59.701110852 (7h32m13.288167063s) [upload=7725]: Waiting for error group managing segments... Aug 7 23:58:59.701144786 (7h32m13.288200996s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:59.701149124 (7h32m13.288205335s) [upload=7725]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:58:59.701151598 (7h32m13.28820782s) [upload=7725]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:58:59.770194014 (7h32m13.357250215s) [upload=7725]: Done uploading segments... Aug 7 23:58:59.770207138 (7h32m13.357263349s) [upload=7725]: Upload finished. err: <nil> Aug 7 23:59:50.057740640 (7h33m3.644796851s) [upload=7734]: Starting upload Aug 7 23:59:50.057782308 (7h33m3.644838529s) [upload=7734]: Done starting upload Aug 7 23:59:50.057821390 (7h33m3.644877601s) [upload=7734]: Uploading segments... Aug 7 23:59:50.057860123 (7h33m3.644916344s) [upload=7734]: Got next segment. Inline: true Aug 7 23:59:50.057863489 (7h33m3.64491971s) [upload=7734]: Waiting for error group managing segments... Aug 7 23:59:50.057892844 (7h33m3.644949065s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:59:50.057896511 (7h33m3.644952723s) [upload=7734]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:59:50.057898685 (7h33m3.644954906s) [upload=7734]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 00:00:20.031073296 (7h33m33.618129506s) [upload=7734]: Done uploading segments... Aug 8 00:00:20.031088353 (7h33m33.618144564s) [upload=7734]: Upload finished. err: metaclient: context deadline exceeded Duration: 258.359702ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [XXXXX..X............................................XXXXXXX.] duration: 1m0.000422639s FAIL (2025-08-08 00:00:19 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000439341s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 969.787935ms FAIL (2025-08-08 00:00:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:4194304 (size:4194304) ======LOGS====== 1345s) [upload=7262]: Got next segment. Inline: false Aug 7 23:31:37.583906637 (7h4m51.170962878s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:31:37.583953414 (7h4m51.171009656s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:31:37.613963041 (7h4m51.201019262s) [seg_pos={0 0}, upload=7262]: Begin upload segment... Aug 7 23:31:37.614008966 (7h4m51.201065176s) [seg_pos={0 0}, upload=7262]: Done begin upload segment. Aug 7 23:31:37.614029103 (7h4m51.201085324s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614108311 (7h4m51.201164542s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614344612 (7h4m51.201400833s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614634754 (7h4m51.201690975s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614919926 (7h4m51.201976137s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.614943150 (7h4m51.201999391s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615146719 (7h4m51.20220294s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615454163 (7h4m51.202510384s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615514677 (7h4m51.202570918s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615688160 (7h4m51.202744381s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.615932236 (7h4m51.202988457s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616184166 (7h4m51.203240378s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616521716 (7h4m51.203577937s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.616643614 (7h4m51.203699855s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621231117 (7h4m51.208287338s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621561163 (7h4m51.208617374s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.621861323 (7h4m51.208917534s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.622156223 (7h4m51.209212444s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Uploading piece... Aug 7 23:31:37.726040884 (7h4m51.313097094s) [upload=7262]: Next returned nil segment Aug 7 23:31:37.726066772 (7h4m51.313122983s) [upload=7262]: Waiting for error group managing segments... Aug 7 23:31:37.736232517 (7h4m51.323288738s) [noise=true , address=45.91.116.4:4959 , piece_id=GCAOGJC2EERNOH7Q4NLCGMX5BJHPPPWAKLJVSS3LK6Z52O6YCYCA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.737656014 (7h4m51.324712245s) [noise=true , address=104.36.82.43:4845 , piece_id=7J5PS3HMVPBRBNGHURPFCUVZIL2XD7IDAMDFSA3PJZ56FD2WV3IA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.739754932 (7h4m51.326811153s) [noise=true , address=104.36.82.22:4515 , piece_id=ULUTYRTCHVD6CVJLXHJ5GQATQPLPRXOVHPRDQJ6MZU2R2P66NDEQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.740628113 (7h4m51.327684344s) [noise=true , address=104.36.82.47:4536 , piece_id=ROPDHSIVOPNHGPWV7DZCZ3GZGJQQB2Q76VBSH53LDXDBEXPTN3NA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741712657 (7h4m51.328768867s) [noise=true , address=104.36.82.45:4939 , piece_id=2HHS67OS52MHE4VGVXJOM6NXWIPYSPHSH5GJN44WU5S7B4S22QNQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.741742653 (7h4m51.328798883s) [noise=true , address=104.36.82.37:4680 , piece_id=P7NUR3BGWKRH5SPUDCLHY6KYZR3NYQ5RJ4NIUJVCRCDRX7CP26FQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.748242274 (7h4m51.335298505s) [noise=true , address=104.36.82.14:4231 , piece_id=AYWBPI67PZRBJ4ZFXKJS5FA7QJQZBE33SSOIK3KEO3SYKCZXNLKA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.750434577 (7h4m51.337490798s) [noise=true , address=104.36.82.23:4468 , piece_id=3SMMMVEKAVPJEYKM3FL5KIVHFEUANKX47MUBOGEVZJPOFU424WGQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.758962364 (7h4m51.346018586s) [noise=true , address=45.91.116.5:4306 , piece_id=3GL3EHAQ4K7LFWOSNFOCD6RVKR5IXLKGLIYKDSUWHQ2ADPGC2OHA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.769693706 (7h4m51.356749917s) [noise=true , address=104.36.82.54:4179 , piece_id=I4456FEQNQF43VTACXRT2EVEFKDAC4ID3V4D562ESLSO3H625KKQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.800540734 (7h4m51.387596975s) [noise=true , address=104.36.82.61:4378 , piece_id=RK2UJMQB7UJR7V5C656ZNU43QWYKK6D7VATSO6ZMOL5JMUG63LAQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:37.924012655 (7h4m51.511068866s) [noise=true , address=104.36.82.59:4327 , piece_id=B4CO5NTMWI3RASH6VSJKGZI6GPJIBNLPL5XM2SS7BQG7WEOQ2WWQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.059695219 (7h4m51.64675144s) [noise=true , address=79.127.201.215:4348 , piece_id=MC5JXBACOI5RB23WMS4ZZM3BQ5OKOBHKQ5MRWT7FHMT7BPE7MT6A, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102515917 (7h4m51.689572138s) [noise=true , address=79.127.205.234:4600 , piece_id=EEIYWE3SXSTJXAM3HSYWJYQJRZCEJTGUF3RSOSIFJLY2BWZS3MMA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: <nil> Aug 7 23:31:38.102541606 (7h4m51.689597827s) [seg_pos={0 0}, upload=7262]: Segment reached optimal threshold of 14 pieces. Aug 7 23:31:38.102658574 (7h4m51.689714795s) [noise=true , address=64.49.13.27:4964 , piece_id=5AN2UI36U4PRJRZAKA7HJ7V62B5PVDRFKRBH65G7RP5YUEZEF3XA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RcgfaeKP3XpZGuSB9GMURCCsjjc79WeZasUL2RRiQVDwxbnAk): protocol: expected piece hash; context canceled; EOF Aug 7 23:31:38.102748111 (7h4m51.689804332s) [noise=true , address=64.49.13.26:4953 , piece_id=HBQDTPFQQCVMKO5F2NQECZU52ZLFWWA52LIL7S7D7B7VGRQR5NAA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvztpAXVj8B4XY7FgaFcsWtqwSxzpV4sRc926EigSpGqnytXE): protocol: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39408->64.49.13.26:4953: use of closed network connection Aug 7 23:31:38.102832158 (7h4m51.689888399s) [noise=true , address=104.36.82.16:4315 , piece_id=TV2LFU35KXULNBE3QMOVAX752IFJTJGQNE633YN5RZVJXOUCNJTQ, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GcyWwKkfavsCsmtq55U72n6XeZKqQiGGYHY9D6UwST8sC14Ac): protocol: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18248->104.36.82.16:4315: use of closed network connection Aug 7 23:31:38.102967200 (7h4m51.690023421s) [noise=true , address=104.36.82.62:4416 , piece_id=LPWXXMNM4JO55TVC7UFVFC6ZRI3FZWEJR3P6LL7ZVDW7PJJUJUWA, seg_pos={0 0}, upload=7262]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: context canceled; context canceled Aug 7 23:31:38.102995132 (7h4m51.690051374s) [seg_pos={0 0}, upload=7262]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 23:31:38.103097553 (7h4m51.690153775s) [upload=7262]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:31:38.103102412 (7h4m51.690158633s) [upload=7262]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:31:38.203633212 (7h4m51.790689433s) [upload=7262]: Done uploading segments... Aug 7 23:31:38.203649102 (7h4m51.790705323s) [upload=7262]: Upload finished. err: <nil> Duration: 969.821117ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.X.X...........................................X.X..XXX...] duration: 1.838018184s FAIL (2025-08-08 00:00:02 UTC) operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.838042639s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.X..........................................X..........] duration: 494.852488ms FAIL (2025-08-07 23:59:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 494.962804ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [X.X..X..X......................................XXX.X.XXX.X..] duration: 200.323516ms FAIL (2025-08-07 23:59:52 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 200.361839ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [XXXXX..X....................................................] duration: 245.230511ms FAIL (2025-08-07 23:59:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 245.253052ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X....XXX.X......................................XXXXX..] duration: 377.566047ms FAIL (2025-08-07 23:59:37 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== , seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522017724 (7h27m3.109073975s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522360804 (7h27m3.109417046s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522463095 (7h27m3.109519336s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522783393 (7h27m3.109839644s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.522884061 (7h27m3.109940302s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523171748 (7h27m3.110228s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523292534 (7h27m3.110348775s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523596000 (7h27m3.110652251s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.523713870 (7h27m3.110770111s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524251423 (7h27m3.111307674s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.524718816 (7h27m3.111775057s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Uploading piece... Aug 7 23:53:49.647138359 (7h27m3.2341946s) [upload=7640]: Next returned nil segment Aug 7 23:53:49.647155511 (7h27m3.234211752s) [upload=7640]: Waiting for error group managing segments... Aug 7 23:53:49.655170752 (7h27m3.242226993s) [noise=true , address=45.91.116.14:4190 , piece_id=PPYDTYVAD7ESR52JJNH64JB22XD5B3WTXTZ7X72X6KCG2FVK5MXA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.655792202 (7h27m3.242848433s) [noise=true , address=45.91.116.5:4297 , piece_id=WYHPPXNXBWTSD4LO2RWXPUN37P7JULRPJSIA3G5KNBYBXXBUDEXQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656163956 (7h27m3.243220187s) [noise=true , address=45.91.116.8:4048 , piece_id=JS3BGIHYWKZYEAXNUCCIYHEDRA2YQMBM2OBPJOUWIAWNEBO6BYFQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.656798870 (7h27m3.243855091s) [noise=true , address=45.91.116.12:4226 , piece_id=YELH2JJ2TFX6A2FUHKFGZFH3RKFQCDUBTA4OBW6B6BSWPJ4FTQSQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.657996216 (7h27m3.245052458s) [noise=true , address=45.91.116.4:4939 , piece_id=LWLSQN6BSOJY4PA7I7HZL5PSRI4DNR6E4GMYWPUB6DF6BLMPU26A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.658644787 (7h27m3.245701017s) [noise=true , address=45.91.116.11:4117 , piece_id=H4RP6BXPN4GRE2TMY5GZEOZ5UKKVDOEAGAS72ZRBIOURV7OBB5MQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.659153586 (7h27m3.246209817s) [noise=true , address=77.90.31.17:4770 , piece_id=QQOG5XJMG2CHWVCGAYWMQMRAHGEKQ5AITZOOOOOVUEFWDFHGBSEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660604835 (7h27m3.247661076s) [noise=true , address=45.91.116.8:4047 , piece_id=SJVTTSCB7BDZRLEJX4WXD4PLXBALDVO2GWIQ5RUI6GAOVMNCERLQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.660941473 (7h27m3.247997694s) [noise=true , address=45.91.116.6:4974 , piece_id=RHPHBCUSDUVI23GGMG2WTVNLMQAAK436N3FXYHPBJ6FV7WOMSFMA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.661457717 (7h27m3.248513959s) [noise=true , address=77.90.31.18:4807 , piece_id=5CCUKS55ROKTHMZA7NKUNYXOTAKYZIDQGDJBSMFM5TSBEFVDJMPA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.662043810 (7h27m3.249100041s) [noise=true , address=104.36.82.6:4874 , piece_id=FZBS75TSUST27CMCEC5UZAAC2ZUNZ5F3PX5SIFHLN72GALS4XNTA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665007553 (7h27m3.252063794s) [noise=true , address=104.36.82.11:4113 , piece_id=N5ABMUNW7Q2MPYAXORNWQMJYIGMHHLYKHZSM7FD265TVN7QPBILQ, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665952497 (7h27m3.253008727s) [noise=true , address=77.90.31.16:4734 , piece_id=AIAJVZAYQTA4OR6WDJ6V2XH3TXBO6FA4VRUCYJGIFXXUO3ZGIT4A, seg_pos={0 0}, upload=7640]: Done uploading piece. err: <nil> Aug 7 23:53:49.665980449 (7h27m3.25303667s) [seg_pos={0 0}, upload=7640]: Segment reached optimal threshold of 13 pieces. Aug 7 23:53:49.666316626 (7h27m3.253372857s) [noise=true , address=104.36.82.14:4221 , piece_id=JKDWKECRRZCOLROOZ5QXNOGIRN3LLLAUGDQRFBPDYBHBV6LTPH7Q, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: rpc: dial tcp 104.36.82.14:4221: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666330082 (7h27m3.253386322s) [noise=true , address=104.36.82.9:4006 , piece_id=6RYH7RTUR45VDJQKTMW6QXWWPPC2FRU2PGECQWGIKUAQOG4DCVEA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: read tcp 5.161.89.252:14750->104.36.82.9:4006: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666492835 (7h27m3.253549076s) [noise=true , address=104.36.82.16:4294 , piece_id=PL7A2YIDU7ZU2F5J4IE37IFPYADS2VNPLY443J46RIBG7J6ZNZJA, seg_pos={0 0}, upload=7640]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: read tcp 5.161.89.252:34080->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 23:53:49.666519565 (7h27m3.253575806s) [seg_pos={0 0}, upload=7640]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 23:53:49.666614813 (7h27m3.253671053s) [upload=7640]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 23:53:49.666620242 (7h27m3.253676484s) [upload=7640]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:57:17.275326827 (7h30m30.862383048s) [upload=7640]: Done uploading segments... Aug 7 23:57:17.275353585 (7h30m30.862409807s) [upload=7640]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:57:17.349138524 (7h30m30.936194735s) [upload=7698]: Starting upload Aug 7 23:57:17.349187294 (7h30m30.936243515s) [upload=7698]: Done starting upload Aug 7 23:57:17.349221879 (7h30m30.93627809s) [upload=7698]: Uploading segments... Aug 7 23:57:17.349404010 (7h30m30.93646022s) [upload=7698]: Got next segment. Inline: false Aug 7 23:57:17.349444765 (7h30m30.936500976s) [upload=7698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:17.349448802 (7h30m30.936505023s) [upload=7698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:57:47.334809460 (7h31m0.921865702s) [upload=7698]: Next segment err: context deadline exceeded Aug 7 23:57:47.334985600 (7h31m0.922041831s) [upload=7698]: Done uploading segments... Aug 7 23:57:47.335006539 (7h31m0.922062781s) [upload=7698]: Upload finished. err: context deadline exceeded Aug 7 23:57:47.412546828 (7h31m0.99960307s) [upload=7705]: Starting upload Aug 7 23:57:47.412626116 (7h31m0.999682357s) [upload=7705]: Done starting upload Aug 7 23:57:47.412754536 (7h31m0.999810777s) [upload=7705]: Uploading segments... Aug 7 23:57:47.413018508 (7h31m1.000074749s) [upload=7705]: Got next segment. Inline: false Aug 7 23:57:47.413067800 (7h31m1.000124042s) [upload=7705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:57:47.413073351 (7h31m1.000129592s) [upload=7705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:17.397856277 (7h31m30.984912519s) [upload=7705]: Next segment err: context deadline exceeded Aug 7 23:58:17.397964409 (7h31m30.98502065s) [upload=7705]: Done uploading segments... Aug 7 23:58:17.397983265 (7h31m30.985039505s) [upload=7705]: Upload finished. err: context deadline exceeded Aug 7 23:58:17.497021948 (7h31m31.084078188s) [upload=7712]: Starting upload Aug 7 23:58:17.497166287 (7h31m31.084222528s) [upload=7712]: Done starting upload Aug 7 23:58:17.497376509 (7h31m31.084432751s) [upload=7712]: Uploading segments... Aug 7 23:58:17.497668814 (7h31m31.084725066s) [upload=7712]: Got next segment. Inline: false Aug 7 23:58:17.497771346 (7h31m31.084827597s) [upload=7712]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:58:17.497783308 (7h31m31.08483955s) [upload=7712]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 23:58:47.474157820 (7h32m1.061214041s) [upload=7712]: Next segment err: context deadline exceeded Aug 7 23:58:47.474221107 (7h32m1.061277319s) [upload=7712]: Done uploading segments... Aug 7 23:58:47.474236366 (7h32m1.061292587s) [upload=7712]: Upload finished. err: context deadline exceeded Duration: 377.595712ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [XXX.X...............................................XX.X....] duration: 166.500853ms FAIL (2025-08-07 23:59:21 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_ObjectCommit Aug 7 23:47:39.988158721 (7h20m53.575214932s) [upload=7536]: Done uploading segments... Aug 7 23:47:39.988176785 (7h20m53.575232995s) [upload=7536]: Upload finished. err: <nil> Aug 7 23:48:17.585904864 (7h21m31.172961076s) [upload=7548]: Starting upload Aug 7 23:48:17.585990855 (7h21m31.173047066s) [upload=7548]: Done starting upload Aug 7 23:48:17.586065835 (7h21m31.173122035s) [upload=7548]: Uploading segments... Aug 7 23:48:17.586102964 (7h21m31.173159205s) [upload=7548]: Got next segment. Inline: true Aug 7 23:48:17.586107262 (7h21m31.173163473s) [upload=7548]: Waiting for error group managing segments... Aug 7 23:48:17.586144451 (7h21m31.173200662s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:48:17.586148969 (7h21m31.17320518s) [upload=7548]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:48:17.586151854 (7h21m31.173208066s) [upload=7548]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:48:17.642355873 (7h21m31.229412094s) [upload=7548]: Done uploading segments... Aug 7 23:48:17.642371582 (7h21m31.229427794s) [upload=7548]: Upload finished. err: <nil> Aug 7 23:49:19.929340015 (7h22m33.516396255s) [upload=7565]: Starting upload Aug 7 23:49:19.929497669 (7h22m33.51655391s) [upload=7565]: Done starting upload Aug 7 23:49:19.929593468 (7h22m33.516649709s) [upload=7565]: Uploading segments... Aug 7 23:49:19.929640947 (7h22m33.516697188s) [upload=7565]: Got next segment. Inline: true Aug 7 23:49:19.929647679 (7h22m33.516703931s) [upload=7565]: Waiting for error group managing segments... Aug 7 23:49:19.929695078 (7h22m33.516751329s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:49:19.929701059 (7h22m33.5167573s) [upload=7565]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:49:19.929704956 (7h22m33.516761197s) [upload=7565]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:49:19.992636681 (7h22m33.579692922s) [upload=7565]: Done uploading segments... Aug 7 23:49:19.992654385 (7h22m33.579710625s) [upload=7565]: Upload finished. err: <nil> Aug 7 23:50:07.176468149 (7h23m20.76352438s) [upload=7576]: Starting upload Aug 7 23:50:07.176537798 (7h23m20.763594019s) [upload=7576]: Done starting upload Aug 7 23:50:07.176592070 (7h23m20.763648311s) [upload=7576]: Uploading segments... Aug 7 23:50:07.176626473 (7h23m20.763682705s) [upload=7576]: Got next segment. Inline: true Aug 7 23:50:07.176631002 (7h23m20.763687213s) [upload=7576]: Waiting for error group managing segments... Aug 7 23:50:07.176664084 (7h23m20.763720295s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:07.176668132 (7h23m20.763724343s) [upload=7576]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:07.176670586 (7h23m20.763726807s) [upload=7576]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:50:07.240851653 (7h23m20.827907864s) [upload=7576]: Done uploading segments... Aug 7 23:50:07.240866771 (7h23m20.827922992s) [upload=7576]: Upload finished. err: <nil> Aug 7 23:50:45.893754262 (7h23m59.480810504s) [upload=7591]: Starting upload Aug 7 23:50:45.893825847 (7h23m59.480882088s) [upload=7591]: Done starting upload Aug 7 23:50:45.893912749 (7h23m59.48096899s) [upload=7591]: Uploading segments... Aug 7 23:50:45.893955949 (7h23m59.4810122s) [upload=7591]: Got next segment. Inline: true Aug 7 23:50:45.893961970 (7h23m59.481018211s) [upload=7591]: Waiting for error group managing segments... Aug 7 23:50:45.894005482 (7h23m59.481061723s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:50:45.894011062 (7h23m59.481067304s) [upload=7591]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:50:45.894014729 (7h23m59.48107097s) [upload=7591]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:05.079621048 (7h24m18.666677259s) [upload=7591]: Done uploading segments... Aug 7 23:51:05.079636297 (7h24m18.666692508s) [upload=7591]: Upload finished. err: <nil> Aug 7 23:51:21.725622238 (7h24m35.312678469s) [upload=7603]: Starting upload Aug 7 23:51:21.725696627 (7h24m35.312752838s) [upload=7603]: Done starting upload Aug 7 23:51:21.725758231 (7h24m35.312814443s) [upload=7603]: Uploading segments... Aug 7 23:51:21.725796774 (7h24m35.312852985s) [upload=7603]: Got next segment. Inline: true Aug 7 23:51:21.725802845 (7h24m35.312859056s) [upload=7603]: Waiting for error group managing segments... Aug 7 23:51:21.725848871 (7h24m35.312905082s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:51:21.725855313 (7h24m35.312911523s) [upload=7603]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:51:21.725859100 (7h24m35.312915351s) [upload=7603]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:51:21.784505948 (7h24m35.371562158s) [upload=7603]: Done uploading segments... Aug 7 23:51:21.784521767 (7h24m35.371577978s) [upload=7603]: Upload finished. err: <nil> Aug 7 23:52:10.285208176 (7h25m23.872264397s) [upload=7615]: Starting upload Aug 7 23:52:10.285263228 (7h25m23.872319449s) [upload=7615]: Done starting upload Aug 7 23:52:10.285310727 (7h25m23.872366937s) [upload=7615]: Uploading segments... Aug 7 23:52:10.285339711 (7h25m23.872395932s) [upload=7615]: Got next segment. Inline: true Aug 7 23:52:10.285343438 (7h25m23.872399649s) [upload=7615]: Waiting for error group managing segments... Aug 7 23:52:10.285391158 (7h25m23.872447379s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:52:10.285395065 (7h25m23.872451276s) [upload=7615]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:52:10.285397770 (7h25m23.872453991s) [upload=7615]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:52:10.400888570 (7h25m23.987944781s) [upload=7615]: Done uploading segments... Aug 7 23:52:10.400904530 (7h25m23.987960741s) [upload=7615]: Upload finished. err: <nil> Aug 7 23:53:11.795016020 (7h26m25.382072262s) [upload=7635]: Starting upload Aug 7 23:53:11.795108102 (7h26m25.382164354s) [upload=7635]: Done starting upload Aug 7 23:53:11.795175588 (7h26m25.382231829s) [upload=7635]: Uploading segments... Aug 7 23:53:11.795254645 (7h26m25.382310886s) [upload=7635]: Got next segment. Inline: true Aug 7 23:53:11.795262130 (7h26m25.38231837s) [upload=7635]: Waiting for error group managing segments... Aug 7 23:53:11.795319296 (7h26m25.382375538s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:53:11.795326170 (7h26m25.38238242s) [upload=7635]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:53:11.795329857 (7h26m25.382386097s) [upload=7635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:54:11.782974743 (7h27m25.370030954s) [upload=7635]: Done uploading segments... Aug 7 23:54:11.782991474 (7h27m25.370047695s) [upload=7635]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:54:11.855031501 (7h27m25.442087722s) [upload=7650]: Starting upload Aug 7 23:54:11.855087627 (7h27m25.442143838s) [upload=7650]: Done starting upload Aug 7 23:54:11.855131608 (7h27m25.442187829s) [upload=7650]: Uploading segments... Aug 7 23:54:11.855163227 (7h27m25.442219448s) [upload=7650]: Got next segment. Inline: true Aug 7 23:54:11.855167005 (7h27m25.442223226s) [upload=7650]: Waiting for error group managing segments... Aug 7 23:54:11.855198905 (7h27m25.442255126s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:54:11.855202661 (7h27m25.442258882s) [upload=7650]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:54:11.855205036 (7h27m25.442261247s) [upload=7650]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:11.844108363 (7h28m25.431164634s) [upload=7650]: Done uploading segments... Aug 7 23:55:11.844129542 (7h28m25.431185763s) [upload=7650]: Upload finished. err: metaclient: context deadline exceeded Aug 7 23:55:11.924363315 (7h28m25.511419527s) [upload=7668]: Starting upload Aug 7 23:55:11.924429198 (7h28m25.511485409s) [upload=7668]: Done starting upload Aug 7 23:55:11.924484862 (7h28m25.511541083s) [upload=7668]: Uploading segments... Aug 7 23:55:11.924519887 (7h28m25.511576098s) [upload=7668]: Got next segment. Inline: true Aug 7 23:55:11.924523564 (7h28m25.511579785s) [upload=7668]: Waiting for error group managing segments... Aug 7 23:55:11.924554131 (7h28m25.511610342s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:11.924558279 (7h28m25.51161449s) [upload=7668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:11.924560643 (7h28m25.511616855s) [upload=7668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:55:12.020746910 (7h28m25.607803121s) [upload=7668]: Done uploading segments... Aug 7 23:55:12.020765404 (7h28m25.607821625s) [upload=7668]: Upload finished. err: <nil> Aug 7 23:55:57.071847012 (7h29m10.658903254s) [upload=7678]: Starting upload Aug 7 23:55:57.071985611 (7h29m10.659041851s) [upload=7678]: Done starting upload Aug 7 23:55:57.072082963 (7h29m10.659139204s) [upload=7678]: Uploading segments... Aug 7 23:55:57.072231269 (7h29m10.65928751s) [upload=7678]: Got next segment. Inline: true Aug 7 23:55:57.072240506 (7h29m10.659296748s) [upload=7678]: Waiting for error group managing segments... Aug 7 23:55:57.072359458 (7h29m10.6594157s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 23:55:57.072367143 (7h29m10.659423384s) [upload=7678]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 23:55:57.072371320 (7h29m10.659427562s) [upload=7678]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 23:56:57.049680168 (7h30m10.636736379s) [upload=7678]: Done uploading segments... Aug 7 23:56:57.049696499 (7h30m10.636752709s) [upload=7678]: Upload finished. err: metaclient: context deadline exceeded Duration:* 166.526331ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)