Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 122685 reports (Page 82 of 2454)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 123.949917ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-27 00:01:04 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 123.970134ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 121.99287ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-27 00:00:35 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 122.025351ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 321.412065ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-27 00:00:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 321.432023ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 124.176288ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-26 23:59:37 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 124.196525ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 138.852275ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-26 23:59:07 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 138.875859ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 117.941191ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-26 23:57:59 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 117.960167ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 141.438338ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-26 23:57:41 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 141.458054ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 120.41881ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-26 23:56:34 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 120.4353ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 296.341908ms remote: "tcp:136.0.77.3:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/4e4aecbb904feb34 FAIL (2026-01-26 23:56:13 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 296.358078ms edge gateway vhost bgp eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 537.387064ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:27:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 537.404788ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [XX..........................................................] duration: 652.416226ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:27:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 652.446411ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.XX.........................................................] duration: 474.885698ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:27:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 474.901948ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 609.048557ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:26:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 609.069305ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..XX........................................................] duration: 467.88004ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:26:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 467.929993ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 333.463135ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:25:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 333.483432ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 648.825629ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:25:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 648.838914ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...XX.......................................................] duration: 490.081769ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:25:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 490.162741ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 342.774427ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:25:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 342.787232ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 846.506096ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:25:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 846.523017ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 355.507853ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:25:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 355.524394ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 811.81283ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:24:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 811.864376ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....XX......................................................] duration: 470.576367ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:24:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 470.604219ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 403.329812ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:24:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 403.341014ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 549.613674ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:23:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 549.626318ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....XX.....................................................] duration: 651.986752ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:23:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 652.009655ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 356.129687ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:23:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 356.157198ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......XX....................................................] duration: 690.054144ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:23:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 690.151184ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 541.764072ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:22:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 541.785472ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 309.460247ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:22:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 309.471589ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......XX...................................................] duration: 507.036711ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:22:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 507.053814ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 396.118812ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:21:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 396.135794ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 740.613313ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:21:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 740.62819ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........XX..................................................] duration: 478.341955ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:21:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 478.362773ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 833.889441ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:20:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 833.907566ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 349.906079ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:20:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 349.923702ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 630.51278ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:20:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 630.531987ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........XX.................................................] duration: 505.309928ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:20:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 505.332629ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 350.362802ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:20:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 350.385486ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 577.383307ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:19:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 577.402474ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........XX................................................] duration: 661.775347ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:19:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 661.790185ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 365.637144ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:19:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 365.669463ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 687.544154ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:19:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 687.566135ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........XX...............................................] duration: 813.124862ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:18:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 813.163044ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 776.119507ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:18:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 776.135909ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 319.045143ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:18:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 319.057095ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............XX..............................................] duration: 729.002908ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:18:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 729.023305ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 293.094328ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:17:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 293.110679ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 595.621707ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:17:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 595.661462ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............XX.............................................] duration: 479.385782ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:17:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.466383ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 316.772318ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:16:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 316.790562ms core uplink us-select-1

1
...
80
81
82
83
84
...
2454