Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
328.36141ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:53:16 UTC)
orbiter warning: context deadline exceeded
Duration: 328.378192ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
500.954418ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:53:00 UTC)
orbiter warning: context deadline exceeded
Duration: 500.976029ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
394.636244ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:52:38 UTC)
orbiter warning: context deadline exceeded
Duration: 394.650801ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
25.494628861s trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:52:37 UTC)
orbiter warning: context deadline exceeded
Duration: 25.494673744s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
370.743639ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:51:25 UTC)
orbiter warning: context deadline exceeded
Duration: 370.763357ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
383.53631ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:50:45 UTC)
orbiter warning: context deadline exceeded
Duration: 383.561317ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
394.198927ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:50:12 UTC)
orbiter warning: context deadline exceeded
Duration: 394.209667ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
364.807119ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:49:14 UTC)
orbiter warning: context deadline exceeded
Duration: 364.823711ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
319.853051ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:48:49 UTC)
orbiter warning: context deadline exceeded
Duration: 319.871516ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
355.324955ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:48:25 UTC)
orbiter warning: context deadline exceeded
Duration: 355.343438ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
374.004892ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:48:08 UTC)
orbiter warning: context deadline exceeded
Duration: 374.027444ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
379.571941ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:47:34 UTC)
orbiter warning: context deadline exceeded
Duration: 379.629678ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
380.322325ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:46:56 UTC)
orbiter warning: context deadline exceeded
Duration: 380.348745ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
403.584701ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:45:49 UTC)
orbiter warning: context deadline exceeded
Duration: 403.603786ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
687.915911ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:44:59 UTC)
orbiter warning: context deadline exceeded
Duration: 687.961526ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
357.33509ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:43:45 UTC)
orbiter warning: context deadline exceeded
Duration: 357.351521ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
408.088293ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:43:29 UTC)
orbiter warning: context deadline exceeded
Duration: 408.110115ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
352.04722ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:43:04 UTC)
orbiter warning: context deadline exceeded
Duration: 352.064072ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
356.834304ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:42:37 UTC)
orbiter warning: context deadline exceeded
Duration: 356.854492ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.109359021s trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:41:56 UTC)
orbiter warning: context deadline exceeded
Duration: 1.109379029s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
389.496409ms trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:40:46 UTC)
orbiter warning: context deadline exceeded
Duration: 389.517979ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000672205s trace:
https://jaeger.dev.storj.io/trace/6c8bbab00f6f74e1
FAIL (2026-02-05 05:40:17 UTC)
orbiter warning: context deadline exceeded
Duration: 30.000686903s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
382.269996ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:05:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 382.298198ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
400.874041ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:04:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 400.891033ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
430.22287ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:03:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 430.244ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
512.883817ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:03:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 512.9068ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
438.812828ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:02:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 438.830331ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
431.601667ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:02:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 431.625259ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
407.653676ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:01:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 407.678382ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
472.793883ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 02:00:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 472.807988ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
365.94616ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:59:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 365.961969ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
493.485647ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:58:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 493.525861ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
488.893006ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:57:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 488.907763ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
414.418183ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:57:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 414.437359ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
386.875088ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:56:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 386.891518ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
415.338695ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:55:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 415.361578ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
362.161279ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:55:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 362.182218ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
369.21118ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:54:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 369.229414ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
409.355285ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:53:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 409.403034ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
411.855266ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:52:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 411.879952ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
501.709591ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:52:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 501.72518ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
342.538791ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:51:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 342.557334ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
376.429393ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:50:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 376.467275ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
461.848753ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:49:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 461.865885ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
436.638348ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:49:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 436.658746ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
469.514055ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:48:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 469.541616ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
481.946942ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:48:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 481.969956ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
460.757061ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:47:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 460.787008ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
481.313377ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:46:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 481.3361ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
404.413448ms trace:
https://jaeger.dev.storj.io/trace/145eaa01a5f69010
FAIL (2026-02-05 01:46:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Feb 5 01:20:27.725290604 (489h24m5.907735604s) [upload=430282]: Starting upload
Feb 5 01:20:27.725401741 (489h24m5.907846731s) [upload=430282]: Done starting upload
Feb 5 01:20:27.725551751 (489h24m5.907996752s) [upload=430282]: Uploading segments...
Feb 5 01:20:27.725761734 (489h24m5.908206735s) [upload=430282]: Got next segment. Inline: false
Feb 5 01:20:27.725830965 (489h24m5.908275954s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 5 01:20:27.725836054 (489h24m5.908281054s) [upload=430282]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 5 01:20:27.746490255 (489h24m5.928935235s) [seg_pos={0 0}, upload=430282]: Begin upload segment...
Feb 5 01:20:27.746528506 (489h24m5.928973486s) [seg_pos={0 0}, upload=430282]: Done begin upload segment.
Feb 5 01:20:27.746559394 (489h24m5.929004375s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746582106 (489h24m5.929027076s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.746902084 (489h24m5.929347054s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747185675 (489h24m5.929630655s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747447925 (489h24m5.929892894s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747710916 (489h24m5.930155896s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.747991050 (489h24m5.93043602s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748271905 (489h24m5.930716885s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748564251 (489h24m5.931009231s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.748853552 (489h24m5.931298521s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.750065517 (489h24m5.932510497s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.751123383 (489h24m5.933568353s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.752191771 (489h24m5.934636741s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.753215824 (489h24m5.935660804s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.754238816 (489h24m5.936683797s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.755442375 (489h24m5.937887355s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Uploading piece...
Feb 5 01:20:27.823680799 (489h24m6.006125769s) [upload=430282]: Next returned nil segment
Feb 5 01:20:27.823690156 (489h24m6.006135126s) [upload=430282]: Waiting for error group managing segments...
Feb 5 01:20:27.831139439 (489h24m6.01358442s) [noise=true , address=45.91.116.11:4152 , piece_id=DID2YTE2RIJS3TPTTVFB5XNSG6FPCRUUECQJ6HXGIX7YCZMAMPQA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831492739 (489h24m6.013937719s) [noise=true , address=45.91.116.11:4120 , piece_id=GJPVGJYHIEC53O6KHPYDBOQ5BJG5X7YINK6TAFT2VD7HJOEF5ZYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.831885924 (489h24m6.014330904s) [noise=true , address=45.91.116.5:4300 , piece_id=6RGY6UY5TEGOJFMVT6YOQNDEVWMQFQ7RS3WVW45AYHJ4PRKKA53A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832032157 (489h24m6.014477137s) [noise=true , address=79.127.219.46:4889 , piece_id=ZUYQSCADY5OFANCA4WKZDZMPTBYTK5KEHDCAZCGVW4G6AAFDSTYQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832341746 (489h24m6.014786716s) [noise=true , address=79.127.219.41:4925 , piece_id=IPHJPSD3CCDDNJ2BNPF3FOB3Q7VLTHVBNUKO5OB246SKBJZ2PBHQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832654280 (489h24m6.01509925s) [noise=true , address=45.91.116.12:4225 , piece_id=TCHGRLU4GXPTXH2Y3LD6L4LL7R3YTRZJWS2T5FIY3SB3A7K2WKMA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.832661003 (489h24m6.015105983s) [noise=true , address=45.91.116.14:4224 , piece_id=AHNHLLQ3ZQMU7VGZF2JMGKFRBLQ6KDDHQXUEW2DPZEMEGCBER5IA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833103049 (489h24m6.015548029s) [noise=true , address=145.79.227.8:4329 , piece_id=ZKMHCRRUUWH7AL56P4FGT5HP4BMXXYMYCZH357ZMBBKIQN3NQ2TQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833259240 (489h24m6.015704211s) [noise=true , address=145.79.227.19:4079 , piece_id=PR4D3RP2O6NZWZ6JBFNSWHW6OKSS2ODBDQKL6HA3HNLB5AVXCRTQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:27.833466969 (489h24m6.015911949s) [noise=true , address=79.127.219.38:4818 , piece_id=VL4PG5OPXYAGRL3OUB4F4ZV5SQL55OZQIFH4WCFIDBAF33E6DFYA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:29.077042539 (489h24m7.259487539s) [noise=true , address=145.79.227.36:4518 , piece_id=FXVEUMUBO3BXHHHCOQMCRMAJZJFBFNT7EAEWWJ34TK3V6CDZVT5A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: <nil>
Feb 5 01:20:57.705588585 (489h24m35.888033555s) [noise=true , address=77.90.31.13:4627 , piece_id=F4BB2XDYPDQYF45AHLQDHWLS2PDCTR4R3HCLTMRNCXSAQV2F4ACQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.705733196 (489h24m35.888178276s) [noise=true , address=77.90.31.12:4590 , piece_id=AW6BS5O7BQTSRLHD5NJDFKYYPXFAVUSCDXAVSHUBQL3ZEGJNEUIQ, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14630->77.90.31.12:4590: use of closed network connection
Feb 5 01:20:57.705919113 (489h24m35.888364083s) [noise=true , address=77.90.31.16:4735 , piece_id=T5ZK3XMZ2RKZAKEQSAO3RFQOOH6PQLHVCHLPIPJPEHFB3QONJR4A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:47824->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706435347 (489h24m35.888880307s) [noise=true , address=77.90.31.19:4841 , piece_id=RFEXJVR4QFCVVEPGNWL2K4GX3RRXFDCHA4UWVDAEBYXFV5AKNHPA, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:29474->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706696186 (489h24m35.889141155s) [noise=true , address=77.90.31.14:4664 , piece_id=3FT3N47EGNZTIQJDYSN6AOEKEOXBZD43YG2AA6MV2FUIGPZU5A3A, seg_pos={0 0}, upload=430282]: Done uploading piece. err: ecclient: upload failed (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9, address:77.90.31.14:4664): protocol: read tcp 5.161.89.252:56630->77.90.31.14:4664: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 5 01:20:57.706722234 (489h24m35.889167204s) [seg_pos={0 0}, upload=430282]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 5 01:20:57.726510627 (489h24m35.908955626s) [upload=430282]: Done uploading segments...
Feb 5 01:20:57.726544360 (489h24m35.90898936s) [upload=430282]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 404.432124ms
core
uplink
us-select-1
|