Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXXXX.XX...................................................] duration: 599.552439ms FAIL (2025-09-22 16:43:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 599.57373ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 512.394004ms FAIL (2025-09-22 16:43:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 512.415604ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXXXX.XX..................................................] duration: 406.442478ms FAIL (2025-09-22 16:42:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 406.46003ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 534.317904ms FAIL (2025-09-22 16:42:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 534.349144ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XXXXXXX.XX.................................................] duration: 734.255308ms FAIL (2025-09-22 16:41:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 734.277189ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 437.417836ms FAIL (2025-09-22 16:41:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 437.448404ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XXXXXXX.XX................................................] duration: 464.468465ms FAIL (2025-09-22 16:41:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 464.557703ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 692.969772ms FAIL (2025-09-22 16:41:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 692.994688ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XXXXXXX.XX...............................................] duration: 410.701532ms FAIL (2025-09-22 16:41:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 410.846884ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [....XXXXXXX.XX..............................................] duration: 582.838871ms FAIL (2025-09-22 16:40:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 582.851785ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 819.603046ms FAIL (2025-09-22 16:40:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 819.628214ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....XXXXXXX.XX.............................................] duration: 442.69268ms FAIL (2025-09-22 16:40:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.706305ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 527.637366ms FAIL (2025-09-22 16:40:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 527.655189ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 955.257654ms FAIL (2025-09-22 16:39:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 955.280796ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [......XXXXXXX.XX............................................] duration: 518.894176ms FAIL (2025-09-22 16:39:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 518.913722ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 736.312712ms FAIL (2025-09-22 16:39:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 736.340284ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 492.541965ms FAIL (2025-09-22 16:38:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 492.559547ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 806.933189ms FAIL (2025-09-22 16:38:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 806.958186ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.......XXXXXXX.XX...........................................] duration: 484.689446ms FAIL (2025-09-22 16:38:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 484.905922ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 453.271813ms FAIL (2025-09-22 16:38:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 453.294365ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 556.450422ms FAIL (2025-09-22 16:38:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 556.472082ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [........XXXXXXX.XX..........................................] duration: 467.780748ms FAIL (2025-09-22 16:38:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 467.79729ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 754.031308ms FAIL (2025-09-22 16:37:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 754.056985ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 505.573769ms FAIL (2025-09-22 16:37:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 505.594918ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.........XXXXXXX.XX.........................................] duration: 479.27086ms FAIL (2025-09-22 16:37:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.294305ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..........XXXXXXX.XX........................................] duration: 457.633654ms FAIL (2025-09-22 16:37:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 457.650045ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 784.909938ms FAIL (2025-09-22 16:37:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 784.930788ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 657.580992ms FAIL (2025-09-22 16:36:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 657.605087ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 875.112608ms FAIL (2025-09-22 16:36:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 875.147593ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...........XXXXXXX.XX.......................................] duration: 715.945139ms FAIL (2025-09-22 16:36:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 715.968344ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [............XXXXXXX.XX......................................] duration: 508.463429ms FAIL (2025-09-22 16:36:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 508.484689ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 796.918036ms FAIL (2025-09-22 16:35:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 796.94173ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 987.187717ms FAIL (2025-09-22 16:35:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 987.207614ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............XXXXXXX.XX.....................................] duration: 423.429486ms FAIL (2025-09-22 16:35:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 423.446538ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 627.890938ms FAIL (2025-09-22 16:34:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 627.917228ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 670.123655ms FAIL (2025-09-22 16:34:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 670.156006ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............XXXXXXX.XX....................................] duration: 548.717866ms FAIL (2025-09-22 16:34:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 548.739347ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 997.434256ms FAIL (2025-09-22 16:34:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 997.457389ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 425.112634ms FAIL (2025-09-22 16:33:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 425.132501ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 987.94515ms FAIL (2025-09-22 16:33:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 987.96109ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 411.6455ms FAIL (2025-09-22 16:33:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 411.665969ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............XXXXXXX.XX...................................] duration: 482.57383ms FAIL (2025-09-22 16:33:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 482.594457ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 888.115153ms FAIL (2025-09-22 16:32:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 888.142283ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................XXXXXXX.XX..................................] duration: 517.230244ms FAIL (2025-09-22 16:32:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 517.25007ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 519.827976ms FAIL (2025-09-22 16:32:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 519.90987ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................XXXXXXX.XX.................................] duration: 439.670283ms FAIL (2025-09-22 16:32:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 439.686533ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 713.229665ms FAIL (2025-09-22 16:31:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 9 (577h50m13.416262728s) [seg_pos={0 0}, upload=508550]: Begin upload segment... Sep 22 15:58:38.063530218 (577h50m13.416315316s) [seg_pos={0 0}, upload=508550]: Done begin upload segment. Sep 22 15:58:38.063561497 (577h50m13.416346594s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.063861729 (577h50m13.416646827s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064091571 (577h50m13.416876668s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064286867 (577h50m13.417071965s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.064500107 (577h50m13.417285215s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065008420 (577h50m13.417793538s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065360400 (577h50m13.418145518s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.065773535 (577h50m13.418558643s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066176291 (577h50m13.418961409s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066506720 (577h50m13.419291828s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.066849383 (577h50m13.419634501s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067154716 (577h50m13.419939834s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.067602916 (577h50m13.420388034s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068042971 (577h50m13.42082809s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068403177 (577h50m13.421188285s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.068850346 (577h50m13.421635454s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069265746 (577h50m13.422050864s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.069568303 (577h50m13.422353421s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Uploading piece... Sep 22 15:58:38.141838308 (577h50m13.494623406s) [upload=508550]: Next returned nil segment Sep 22 15:58:38.141848227 (577h50m13.494633325s) [upload=508550]: Waiting for error group managing segments... Sep 22 15:58:38.152429755 (577h50m13.505214853s) [noise=true , address=77.90.31.15:4709 , piece_id=UZ3PUQJURZ3XSZOIAVQFX6P7KFMOIMYNE7TDFKBPX47DQJYJUSKA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.153164493 (577h50m13.505949621s) [noise=true , address=77.90.31.21:4882 , piece_id=MSE32EW54QZWETSWIQE6KET7A3NHY7LZRPV4KFINOLJS4GRXH2GA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:38.154207529 (577h50m13.506992627s) [noise=true , address=77.90.31.16:4750 , piece_id=3YRKFZHCNODNP2QU7KEUG5CVCQWH7YJVFHL77V7J6J3MGD4BBW2Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:42.055718428 (577h50m17.408503526s) [noise=true , address=145.79.227.46:4622 , piece_id=HFQ56PBGIA5P6APE6UV7DLEHZS6FYLNVSBBYIZJCAGQAV5E3U3YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:43.913368294 (577h50m19.266153392s) [noise=true , address=145.79.227.38:4000 , piece_id=WZG4BO5DIURMDRWREXDDVOIQFC3O72YRUZKTT3Z2A6FKF2HKPUNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:45.252468619 (577h50m20.605253747s) [noise=true , address=145.79.227.23:4580 , piece_id=KHBAAIV447MRLTMFLR7CESSFJ42CGOPHPXEGYBGP4Q6UGUGYMVTA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:46.045169364 (577h50m21.397954463s) [noise=true , address=145.79.227.2:4133 , piece_id=H7FUFOGJO32ESH6HBBK5GY5ELVU5RD3WHN7ADZFZQKYJ5OLB33CQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:58:54.509372826 (577h50m29.862157934s) [noise=true , address=145.79.227.17:4000 , piece_id=QP2BDLWP5RHDN6ADS4R4MIK3MXTBST2XWBOY52L7KMAVMEMUFASQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:00.589899602 (577h50m35.94268471s) [noise=true , address=145.79.227.49:4343 , piece_id=ZHZ2TLE524NNFTWBVZRVJOQFA3E52BPOLRD5O4G5CDTGCLQXQZFQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:02.947508340 (577h50m38.300293448s) [noise=true , address=145.79.227.52:4476 , piece_id=YSZSZX64GJIZ3NMLVSP2HUSC337HRLCBIZDCWYDFGLU2TSJF5PRA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:07.378570652 (577h50m42.73135576s) [noise=true , address=145.79.227.7:4307 , piece_id=E2X4ECTEUU5SX5D33N57ROQTJDYX3ZWNQAX2W4BX2QFGAOSAA7TQ, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:09.627000384 (577h50m44.979785522s) [noise=true , address=145.79.227.6:4264 , piece_id=H47BO3HBPYMCA6H23TOIJOQDZU5GWANI2NCI5NBSCKWB7ALCCE3Q, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:33.191097524 (577h51m8.543882632s) [noise=true , address=145.79.227.12:4772 , piece_id=IXETXIOMWX4V4E2PPEYHU5L4NB3F7XYAUPSKWBVR2WMV7XW6AYWA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: <nil> Sep 22 15:59:38.021980821 (577h51m13.374765929s) [noise=true , address=145.79.227.4:4203 , piece_id=WRPAZ66AAZ6L7N5TMH7BQW25WE2SRRQFTMOGJRYCFBVZMZAYS5YA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:18cNSsgqn3n6z5LxRcVCRzDzfFDpvptV1MKh6NdNRLW4bg9fWM, address:145.79.227.4:4203): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022102409 (577h51m13.374887537s) [noise=true , address=145.79.227.30:4767 , piece_id=PYSUDVVXLF77V4HPULHSCBJA5S5UIPYG5OQQ37BAFEKVS7YQZ2VA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:12sfaYCUhNCAyQAPu1jac5dRWR35Qqc4ZcKg75mp7nVQak295ug, address:145.79.227.30:4767): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:38.022147594 (577h51m13.374932702s) [noise=true , address=145.79.227.4:4194 , piece_id=I3DLRS42BRA7R2YDQL5KL3E72GNQL2EVV2JTSR45UOAPLHR6NTNA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1u6tGwzk56SwgUqzpyRFLNTTysUtk6C2Ls77UkjFryeuF9D8pV, address:145.79.227.4:4194): protocol: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47298->145.79.227.4:4194: use of closed network connection Sep 22 15:59:38.022217905 (577h51m13.375003013s) [noise=true , address=145.79.227.16:4939 , piece_id=6RVLFHX736NFVHCYTU35YCAVZSAQDDMNX4IBWQQYRM5WUE5XQU7A, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1mRmp3HhD95PruS4AF8soRsSXNYUeb4mJE7y52FvoeN9D3Gu4F, address:145.79.227.16:4939): protocol: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10158->145.79.227.16:4939: use of closed network connection Sep 22 15:59:38.022247200 (577h51m13.375032298s) [noise=true , address=145.79.227.28:4740 , piece_id=7OBAY6T4NQM6A3VFSMLWL37R5NHWF267NYMUKQJN45FXSK5LWTYA, seg_pos={0 0}, upload=508550]: Done uploading piece. err: ecclient: upload failed (node:1A2tCUpDAzrSzhUfBf6GMsZdpY6SPkRHWnqrWiiy7uNGfzcYbq, address:145.79.227.28:4740): protocol: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41070->145.79.227.28:4740: use of closed network connection Sep 22 15:59:38.022260535 (577h51m13.375045643s) [seg_pos={0 0}, upload=508550]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 15:59:38.046558371 (577h51m13.399343498s) [upload=508550]: Done uploading segments... Sep 22 15:59:38.046584219 (577h51m13.399369347s) [upload=508550]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 713.262096ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 446.041912ms FAIL (2025-09-22 16:31:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 446.06739ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 986.636231ms FAIL (2025-09-22 16:31:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== iece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165766906 (577h50m28.518552034s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.165846755 (577h50m28.518631874s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166356251 (577h50m28.519141379s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166376940 (577h50m28.519162069s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166744760 (577h50m28.519529888s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.166953762 (577h50m28.5197389s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167152305 (577h50m28.519937433s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167457428 (577h50m28.520242556s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167514835 (577h50m28.520299964s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.167882465 (577h50m28.520667593s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.168454006 (577h50m28.521239135s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Uploading piece... Sep 22 15:58:53.283262501 (577h50m28.636047599s) [upload=508554]: Next returned nil segment Sep 22 15:58:53.283272560 (577h50m28.636057668s) [upload=508554]: Waiting for error group managing segments... Sep 22 15:58:53.292785532 (577h50m28.645570631s) [noise=true , address=45.91.116.12:4227 , piece_id=GT32ZMBZFHPRNXN4UEPSX7OHELYLVNOQRMHDCZSYPIBWSHXJHKBQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.293549977 (577h50m28.646335085s) [noise=true , address=45.91.116.6:4974 , piece_id=BIF3QQM3FBDJHCBFSO6YS6ZWLVNM3RTZLF4EPVHXVUA3MHCJMMQA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294582142 (577h50m28.64736724s) [noise=true , address=45.91.116.9:4262 , piece_id=PA3UFE2BH64EZ7PKZ3NSLNE3K5BQDG7X6P2M5CVJRFIECGXQKBXQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.294998735 (577h50m28.647783832s) [noise=true , address=45.91.116.10:4083 , piece_id=XSDQAML5EIHDCKGGP5OODYCKH57DIR7RJVWN2XT3DEF3TCWUOONA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:58:53.299822101 (577h50m28.652607199s) [noise=true , address=77.90.31.17:4771 , piece_id=3DLK4KZYGVQ7YNTJX2XNW44OLTO4YLT5GZPT62JKVFKUR4HGGDZQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:15.582328150 (577h50m50.935113248s) [noise=true , address=145.79.227.18:4006 , piece_id=N7R2WJBV5QTUYHIWO2OTP64NAFKNR5524QZJ5RF6ZL5BGIYZDUHA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: <nil> Sep 22 15:59:23.132335413 (577h50m58.485120521s) [noise=true , address=145.79.227.3:4151 , piece_id=5O7IKJD52ILOKPOKKPFROT4HOBCLIZXNQORKFENLXG4NEKIFLQXA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132401396 (577h50m58.485186504s) [noise=true , address=145.79.227.3:4149 , piece_id=4XDAA7TMSHGW7KVVJQAGIOCAMWZUTQKSJFGBFO4K325VZWEI7JGA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: expected piece hash; context deadline exceeded; EOF Sep 22 15:59:23.132541449 (577h50m58.485326557s) [noise=true , address=145.79.227.5:4221 , piece_id=6BX5KTYGEIMX5FL3XALQM6U3O7AJNJ6TY4GXIL6PXWYU22GMCGWA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12430->145.79.227.5:4221: use of closed network connection Sep 22 15:59:23.132589599 (577h50m58.485374697s) [noise=true , address=145.79.227.6:4257 , piece_id=MUTKID5GNUWEKJMJKUVTE64L6UPC5Z6C2IZEO3PA7D3KYLQL2A7A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:37328->145.79.227.6:4257: use of closed network connection Sep 22 15:59:23.132633662 (577h50m58.48541876s) [noise=true , address=145.79.227.2:4114 , piece_id=WL6TUPDZUCODM7FNGA6QCX3DBTHST53W46TBWO7DZZ3LSUWMKQWQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47042->145.79.227.2:4114: use of closed network connection Sep 22 15:59:23.132668778 (577h50m58.485453876s) [noise=true , address=145.79.227.19:4079 , piece_id=XHUEMVHTTYSVW5MPGMRNPYXUDW77O7LJ3HIYAKVRO5YD4SD7EFPQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:13366->145.79.227.19:4079: use of closed network connection Sep 22 15:59:23.132732587 (577h50m58.485517685s) [noise=true , address=145.79.227.13:4803 , piece_id=WBDRKFCF53ZG5J6O7XFAQ6UVFMUGDVORI4ESSVVDBGTDQEMGCLAA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45026->145.79.227.13:4803: use of closed network connection Sep 22 15:59:23.132741054 (577h50m58.485526162s) [noise=true , address=145.79.227.11:4732 , piece_id=3V33DUXWVU43F5Q7OCWZ2ANBVWGIPUDD3OWNFTCPSC4MTZVLI5DA, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:18250->145.79.227.11:4732: use of closed network connection Sep 22 15:59:23.132758296 (577h50m58.485543394s) [noise=true , address=145.79.227.5:4223 , piece_id=EAPJGEVQMTI5JIAVVQ6SIYIOKREMUABUXQLEGMGLDYHXKFYBTG6A, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:14614->145.79.227.5:4223: use of closed network connection Sep 22 15:59:23.132771250 (577h50m58.485556348s) [noise=true , address=145.79.227.14:4839 , piece_id=357J5DY57NU7CK3X3EX6DPGUVCAIMLMYC3ZS7DAADHKEG4NNDIEQ, seg_pos={0 0}, upload=508554]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61224->145.79.227.14:4839: use of closed network connection Sep 22 15:59:23.132791478 (577h50m58.485576586s) [seg_pos={0 0}, upload=508554]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 22 15:59:23.146345612 (577h50m58.4991307s) [upload=508554]: Done uploading segments... Sep 22 15:59:23.146361672 (577h50m58.49914677s) [upload=508554]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 986.663853ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................XXXXXXX.XX................................] duration: 467.874576ms FAIL (2025-09-22 16:31:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 467.896968ms edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)