Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 116995 reports (Page 471 of 2340)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 412.169246ms FAIL (2025-10-31 22:19:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 412.191526ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................X........................XXXX..X.] duration: 25.115585679s FAIL (2025-10-31 22:18:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.115598102s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 893.006369ms FAIL (2025-10-31 22:18:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 893.25837ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 695.050669ms FAIL (2025-10-31 22:18:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 695.07196ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.879011464s FAIL (2025-10-31 22:18:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.879027544s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 669.048828ms FAIL (2025-10-31 22:18:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 669.061701ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................X........................XXXX..X] duration: 30.001328228s FAIL (2025-10-31 22:18:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001342714s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 4.54981846s FAIL (2025-10-31 22:17:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.549833418s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.137974235s FAIL (2025-10-31 22:17:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.137991268s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.453132s FAIL (2025-10-31 22:17:28 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.453149673s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................X........................XXXX..] duration: 22.332428333s FAIL (2025-10-31 22:17:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 22.332446196s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 674.635765ms FAIL (2025-10-31 22:17:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 674.661102ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 968.16812ms FAIL (2025-10-31 22:17:01 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 968.192465ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 8.402922733s FAIL (2025-10-31 22:16:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.402946317s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.............................X........................XXXX.] duration: 11.256816038s FAIL (2025-10-31 22:16:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.256834352s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.622148661s FAIL (2025-10-31 22:16:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 1.622171274s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.150802849s FAIL (2025-10-31 22:16:06 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.150820302s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.605940038s FAIL (2025-10-31 22:15:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 1.605957881s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X...............................X...........................] duration: 17.818123209s FAIL (2025-10-31 22:15:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.818284201s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.............................X........................XXXX] duration: 30.000559543s FAIL (2025-10-31 22:15:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000576666s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.58608213s FAIL (2025-10-31 22:15:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.586100073s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.188890503s FAIL (2025-10-31 22:14:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.188912615s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X...............................X..........................] duration: 9.708916098s FAIL (2025-10-31 22:14:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.708930354s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.............................X........................XXX] duration: 30.000401622s FAIL (2025-10-31 22:14:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000414456s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.583071357s FAIL (2025-10-31 22:14:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.583090291s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.097142243s FAIL (2025-10-31 22:14:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.097197335s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.370226463s FAIL (2025-10-31 22:14:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== CVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109647425 (106h13m50.81059232s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109869402 (106h13m50.810814287s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109978557 (106h13m50.810923441s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110164685 (106h13m50.81110958s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110399856 (106h13m50.811344751s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110450070 (106h13m50.811394965s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110726599 (106h13m50.811671484s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110789657 (106h13m50.811734543s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111003198 (106h13m50.811948093s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111150555 (106h13m50.812095439s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111291028 (106h13m50.812235923s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111562457 (106h13m50.812507342s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111573357 (106h13m50.812518252s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.185273470 (106h13m50.886218365s) [upload=93483]: Next returned nil segment Oct 31 21:27:24.185306943 (106h13m50.886251838s) [upload=93483]: Waiting for error group managing segments... Oct 31 21:27:24.198842094 (106h13m50.899786989s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202566047 (106h13m50.903510942s) [noise=true , address=145.79.227.7:4326 , piece_id=UWKMPOGPLOEQ7RIN6HB4CPL6I2UBLKYZ4IABXKX35WDPO2LRAOMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202751024 (106h13m50.903695909s) [noise=true , address=145.79.227.49:4368 , piece_id=5KFAGAIUHS45PRRPHM53IWSCVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.206213055 (106h13m50.90715795s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.224640360 (106h13m50.925585246s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.225634635 (106h13m50.92657953s) [noise=true , address=145.79.227.35:4501 , piece_id=HPFY3UD3P3UKMH3F352UTYVE4VYJQHJEZE4O6KMN4DA6H2H7VEMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.232531017 (106h13m50.933475902s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:13.947408711 (106h14m40.648353626s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:21.276856063 (106h14m47.977800958s) [noise=true , address=45.91.116.11:4137 , piece_id=UQEYQVYU5NQLEMQJV2U33DEOOCYYO6DNX4NV2ZSK2E5U3OEIWWGA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:24.069871428 (106h14m50.770816313s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: expected piece hash; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; EOF Oct 31 21:28:24.069950206 (106h14m50.770895091s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: read tcp 5.161.89.252:58260->77.90.31.11:4568: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.069974752 (106h14m50.770919647s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1YTkM1w6HKKyw8z3eWomLMtJuFxsswqjZ5TmGyBeiZJE9QrFoB, address:45.91.116.4:4947): protocol: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection Oct 31 21:28:24.070022812 (106h14m50.770967707s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1WWDBuhCg17uYobyN1WCxT6TdHXjHg9NknseutbiCpKdGo1Nd1, address:45.91.116.8:4076): protocol: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection Oct 31 21:28:24.070054221 (106h14m50.770999106s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection Oct 31 21:28:24.070113412 (106h14m50.771058297s) [noise=true , address=77.90.31.10:4546 , piece_id=OZQE54VEKSLC6AWZVREEH4ES6XFLBE6DI5C7QAP2Y6264AGXTPNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK, address:77.90.31.10:4546): protocol: expected piece hash; context deadline exceeded; EOF Oct 31 21:28:24.070129752 (106h14m50.771074638s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12oSWAY52TJGVyZk4HTfifZBHQv42QsrAsk1Ks9PH6FoEmAZiN3, address:77.90.31.12:4621): protocol: read tcp 5.161.89.252:61442->77.90.31.12:4621: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.070184245 (106h14m50.77112914s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM, address:77.90.31.16:4737): protocol: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection Oct 31 21:28:24.070274645 (106h14m50.77121954s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1qC4ZTAobMwMUu7DQ34re4G4TV1jGEHg2UBXbaEM4k12EggNJW, address:45.91.116.8:4054): protocol: context deadline exceeded; context deadline exceeded Oct 31 21:28:24.070293379 (106h14m50.771238265s) [seg_pos={0 0}, upload=93483]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Oct 31 21:28:24.087202136 (106h14m50.788147031s) [upload=93483]: Done uploading segments... Oct 31 21:28:24.087224858 (106h14m50.788169744s) [upload=93483]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1.37024583s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.............................X........................XX] duration: 30.000679342s FAIL (2025-10-31 22:14:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000697044s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.074632936s FAIL (2025-10-31 22:13:57 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.074655789s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.419384276s FAIL (2025-10-31 22:13:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== CVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109647425 (106h13m50.81059232s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109869402 (106h13m50.810814287s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109978557 (106h13m50.810923441s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110164685 (106h13m50.81110958s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110399856 (106h13m50.811344751s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110450070 (106h13m50.811394965s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110726599 (106h13m50.811671484s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110789657 (106h13m50.811734543s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111003198 (106h13m50.811948093s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111150555 (106h13m50.812095439s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111291028 (106h13m50.812235923s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111562457 (106h13m50.812507342s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111573357 (106h13m50.812518252s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.185273470 (106h13m50.886218365s) [upload=93483]: Next returned nil segment Oct 31 21:27:24.185306943 (106h13m50.886251838s) [upload=93483]: Waiting for error group managing segments... Oct 31 21:27:24.198842094 (106h13m50.899786989s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202566047 (106h13m50.903510942s) [noise=true , address=145.79.227.7:4326 , piece_id=UWKMPOGPLOEQ7RIN6HB4CPL6I2UBLKYZ4IABXKX35WDPO2LRAOMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202751024 (106h13m50.903695909s) [noise=true , address=145.79.227.49:4368 , piece_id=5KFAGAIUHS45PRRPHM53IWSCVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.206213055 (106h13m50.90715795s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.224640360 (106h13m50.925585246s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.225634635 (106h13m50.92657953s) [noise=true , address=145.79.227.35:4501 , piece_id=HPFY3UD3P3UKMH3F352UTYVE4VYJQHJEZE4O6KMN4DA6H2H7VEMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.232531017 (106h13m50.933475902s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:13.947408711 (106h14m40.648353626s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:21.276856063 (106h14m47.977800958s) [noise=true , address=45.91.116.11:4137 , piece_id=UQEYQVYU5NQLEMQJV2U33DEOOCYYO6DNX4NV2ZSK2E5U3OEIWWGA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:24.069871428 (106h14m50.770816313s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: expected piece hash; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; EOF Oct 31 21:28:24.069950206 (106h14m50.770895091s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: read tcp 5.161.89.252:58260->77.90.31.11:4568: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.069974752 (106h14m50.770919647s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1YTkM1w6HKKyw8z3eWomLMtJuFxsswqjZ5TmGyBeiZJE9QrFoB, address:45.91.116.4:4947): protocol: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection Oct 31 21:28:24.070022812 (106h14m50.770967707s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1WWDBuhCg17uYobyN1WCxT6TdHXjHg9NknseutbiCpKdGo1Nd1, address:45.91.116.8:4076): protocol: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection Oct 31 21:28:24.070054221 (106h14m50.770999106s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection Oct 31 21:28:24.070113412 (106h14m50.771058297s) [noise=true , address=77.90.31.10:4546 , piece_id=OZQE54VEKSLC6AWZVREEH4ES6XFLBE6DI5C7QAP2Y6264AGXTPNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK, address:77.90.31.10:4546): protocol: expected piece hash; context deadline exceeded; EOF Oct 31 21:28:24.070129752 (106h14m50.771074638s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12oSWAY52TJGVyZk4HTfifZBHQv42QsrAsk1Ks9PH6FoEmAZiN3, address:77.90.31.12:4621): protocol: read tcp 5.161.89.252:61442->77.90.31.12:4621: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.070184245 (106h14m50.77112914s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM, address:77.90.31.16:4737): protocol: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection Oct 31 21:28:24.070274645 (106h14m50.77121954s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1qC4ZTAobMwMUu7DQ34re4G4TV1jGEHg2UBXbaEM4k12EggNJW, address:45.91.116.8:4054): protocol: context deadline exceeded; context deadline exceeded Oct 31 21:28:24.070293379 (106h14m50.771238265s) [seg_pos={0 0}, upload=93483]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Oct 31 21:28:24.087202136 (106h14m50.788147031s) [upload=93483]: Done uploading segments... Oct 31 21:28:24.087224858 (106h14m50.788169744s) [upload=93483]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 2.419404836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X.X...............................X.........................] duration: 10.333612709s FAIL (2025-10-31 22:13:48 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.333796814s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [X.................................X.........................] duration: 1.398588659s FAIL (2025-10-31 22:13:40 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.398606952s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.............................X........................X] duration: 30.000647262s FAIL (2025-10-31 22:13:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000659684s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 3.055228219s FAIL (2025-10-31 22:13:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.055255339s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.X.................................X........................] duration: 684.828267ms FAIL (2025-10-31 22:13:02 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 684.849448ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.................................X.........................] duration: 1.904643199s FAIL (2025-10-31 22:12:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 1.904671151s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.663674442s FAIL (2025-10-31 22:12:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== CVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109647425 (106h13m50.81059232s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109869402 (106h13m50.810814287s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109978557 (106h13m50.810923441s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110164685 (106h13m50.81110958s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110399856 (106h13m50.811344751s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110450070 (106h13m50.811394965s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110726599 (106h13m50.811671484s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110789657 (106h13m50.811734543s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111003198 (106h13m50.811948093s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111150555 (106h13m50.812095439s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111291028 (106h13m50.812235923s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111562457 (106h13m50.812507342s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111573357 (106h13m50.812518252s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.185273470 (106h13m50.886218365s) [upload=93483]: Next returned nil segment Oct 31 21:27:24.185306943 (106h13m50.886251838s) [upload=93483]: Waiting for error group managing segments... Oct 31 21:27:24.198842094 (106h13m50.899786989s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202566047 (106h13m50.903510942s) [noise=true , address=145.79.227.7:4326 , piece_id=UWKMPOGPLOEQ7RIN6HB4CPL6I2UBLKYZ4IABXKX35WDPO2LRAOMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202751024 (106h13m50.903695909s) [noise=true , address=145.79.227.49:4368 , piece_id=5KFAGAIUHS45PRRPHM53IWSCVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.206213055 (106h13m50.90715795s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.224640360 (106h13m50.925585246s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.225634635 (106h13m50.92657953s) [noise=true , address=145.79.227.35:4501 , piece_id=HPFY3UD3P3UKMH3F352UTYVE4VYJQHJEZE4O6KMN4DA6H2H7VEMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.232531017 (106h13m50.933475902s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:13.947408711 (106h14m40.648353626s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:21.276856063 (106h14m47.977800958s) [noise=true , address=45.91.116.11:4137 , piece_id=UQEYQVYU5NQLEMQJV2U33DEOOCYYO6DNX4NV2ZSK2E5U3OEIWWGA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:24.069871428 (106h14m50.770816313s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: expected piece hash; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; EOF Oct 31 21:28:24.069950206 (106h14m50.770895091s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: read tcp 5.161.89.252:58260->77.90.31.11:4568: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.069974752 (106h14m50.770919647s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1YTkM1w6HKKyw8z3eWomLMtJuFxsswqjZ5TmGyBeiZJE9QrFoB, address:45.91.116.4:4947): protocol: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection Oct 31 21:28:24.070022812 (106h14m50.770967707s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1WWDBuhCg17uYobyN1WCxT6TdHXjHg9NknseutbiCpKdGo1Nd1, address:45.91.116.8:4076): protocol: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection Oct 31 21:28:24.070054221 (106h14m50.770999106s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection Oct 31 21:28:24.070113412 (106h14m50.771058297s) [noise=true , address=77.90.31.10:4546 , piece_id=OZQE54VEKSLC6AWZVREEH4ES6XFLBE6DI5C7QAP2Y6264AGXTPNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK, address:77.90.31.10:4546): protocol: expected piece hash; context deadline exceeded; EOF Oct 31 21:28:24.070129752 (106h14m50.771074638s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12oSWAY52TJGVyZk4HTfifZBHQv42QsrAsk1Ks9PH6FoEmAZiN3, address:77.90.31.12:4621): protocol: read tcp 5.161.89.252:61442->77.90.31.12:4621: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.070184245 (106h14m50.77112914s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM, address:77.90.31.16:4737): protocol: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection Oct 31 21:28:24.070274645 (106h14m50.77121954s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1qC4ZTAobMwMUu7DQ34re4G4TV1jGEHg2UBXbaEM4k12EggNJW, address:45.91.116.8:4054): protocol: context deadline exceeded; context deadline exceeded Oct 31 21:28:24.070293379 (106h14m50.771238265s) [seg_pos={0 0}, upload=93483]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Oct 31 21:28:24.087202136 (106h14m50.788147031s) [upload=93483]: Done uploading segments... Oct 31 21:28:24.087224858 (106h14m50.788169744s) [upload=93483]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1.663695331s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..X.................................X.......................] duration: 1.032158574s FAIL (2025-10-31 22:12:43 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.032200822s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.X.X...............................X........................] duration: 5.149339218s FAIL (2025-10-31 22:12:40 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.149349858s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [...X.................................X......................] duration: 564.033302ms FAIL (2025-10-31 22:12:16 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 564.055524ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.............................X........................] duration: 10.35010781s FAIL (2025-10-31 22:11:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.35016606s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.353429013s FAIL (2025-10-31 22:11:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== CVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109647425 (106h13m50.81059232s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109869402 (106h13m50.810814287s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109978557 (106h13m50.810923441s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110164685 (106h13m50.81110958s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110399856 (106h13m50.811344751s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110450070 (106h13m50.811394965s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110726599 (106h13m50.811671484s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110789657 (106h13m50.811734543s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111003198 (106h13m50.811948093s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111150555 (106h13m50.812095439s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111291028 (106h13m50.812235923s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111562457 (106h13m50.812507342s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111573357 (106h13m50.812518252s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.185273470 (106h13m50.886218365s) [upload=93483]: Next returned nil segment Oct 31 21:27:24.185306943 (106h13m50.886251838s) [upload=93483]: Waiting for error group managing segments... Oct 31 21:27:24.198842094 (106h13m50.899786989s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202566047 (106h13m50.903510942s) [noise=true , address=145.79.227.7:4326 , piece_id=UWKMPOGPLOEQ7RIN6HB4CPL6I2UBLKYZ4IABXKX35WDPO2LRAOMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202751024 (106h13m50.903695909s) [noise=true , address=145.79.227.49:4368 , piece_id=5KFAGAIUHS45PRRPHM53IWSCVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.206213055 (106h13m50.90715795s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.224640360 (106h13m50.925585246s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.225634635 (106h13m50.92657953s) [noise=true , address=145.79.227.35:4501 , piece_id=HPFY3UD3P3UKMH3F352UTYVE4VYJQHJEZE4O6KMN4DA6H2H7VEMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.232531017 (106h13m50.933475902s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:13.947408711 (106h14m40.648353626s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:21.276856063 (106h14m47.977800958s) [noise=true , address=45.91.116.11:4137 , piece_id=UQEYQVYU5NQLEMQJV2U33DEOOCYYO6DNX4NV2ZSK2E5U3OEIWWGA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:24.069871428 (106h14m50.770816313s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: expected piece hash; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; EOF Oct 31 21:28:24.069950206 (106h14m50.770895091s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: read tcp 5.161.89.252:58260->77.90.31.11:4568: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.069974752 (106h14m50.770919647s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1YTkM1w6HKKyw8z3eWomLMtJuFxsswqjZ5TmGyBeiZJE9QrFoB, address:45.91.116.4:4947): protocol: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection Oct 31 21:28:24.070022812 (106h14m50.770967707s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1WWDBuhCg17uYobyN1WCxT6TdHXjHg9NknseutbiCpKdGo1Nd1, address:45.91.116.8:4076): protocol: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection Oct 31 21:28:24.070054221 (106h14m50.770999106s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection Oct 31 21:28:24.070113412 (106h14m50.771058297s) [noise=true , address=77.90.31.10:4546 , piece_id=OZQE54VEKSLC6AWZVREEH4ES6XFLBE6DI5C7QAP2Y6264AGXTPNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK, address:77.90.31.10:4546): protocol: expected piece hash; context deadline exceeded; EOF Oct 31 21:28:24.070129752 (106h14m50.771074638s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12oSWAY52TJGVyZk4HTfifZBHQv42QsrAsk1Ks9PH6FoEmAZiN3, address:77.90.31.12:4621): protocol: read tcp 5.161.89.252:61442->77.90.31.12:4621: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.070184245 (106h14m50.77112914s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM, address:77.90.31.16:4737): protocol: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection Oct 31 21:28:24.070274645 (106h14m50.77121954s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1qC4ZTAobMwMUu7DQ34re4G4TV1jGEHg2UBXbaEM4k12EggNJW, address:45.91.116.8:4054): protocol: context deadline exceeded; context deadline exceeded Oct 31 21:28:24.070293379 (106h14m50.771238265s) [seg_pos={0 0}, upload=93483]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Oct 31 21:28:24.087202136 (106h14m50.788147031s) [upload=93483]: Done uploading segments... Oct 31 21:28:24.087224858 (106h14m50.788169744s) [upload=93483]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 1.353477875s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.................................X........................] duration: 1.157302528s FAIL (2025-10-31 22:11:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 1.157327044s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..X.X...............................X.......................] duration: 1.58180471s FAIL (2025-10-31 22:11:46 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.581823455s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [....X.................................X.....................] duration: 1.207160991s FAIL (2025-10-31 22:11:33 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.207188653s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 959.040674ms FAIL (2025-10-31 22:11:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== CVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109647425 (106h13m50.81059232s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109869402 (106h13m50.810814287s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.109978557 (106h13m50.810923441s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110164685 (106h13m50.81110958s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110399856 (106h13m50.811344751s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110450070 (106h13m50.811394965s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110726599 (106h13m50.811671484s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.110789657 (106h13m50.811734543s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111003198 (106h13m50.811948093s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111150555 (106h13m50.812095439s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111291028 (106h13m50.812235923s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111562457 (106h13m50.812507342s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.111573357 (106h13m50.812518252s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Uploading piece... Oct 31 21:27:24.185273470 (106h13m50.886218365s) [upload=93483]: Next returned nil segment Oct 31 21:27:24.185306943 (106h13m50.886251838s) [upload=93483]: Waiting for error group managing segments... Oct 31 21:27:24.198842094 (106h13m50.899786989s) [noise=true , address=145.79.227.11:4752 , piece_id=DK2ALNHAYCSZYC7GNSPADHOUYMX2G6SI7VOEE564XBUONMXESSEA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202566047 (106h13m50.903510942s) [noise=true , address=145.79.227.7:4326 , piece_id=UWKMPOGPLOEQ7RIN6HB4CPL6I2UBLKYZ4IABXKX35WDPO2LRAOMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.202751024 (106h13m50.903695909s) [noise=true , address=145.79.227.49:4368 , piece_id=5KFAGAIUHS45PRRPHM53IWSCVORDO4VKXA7QG2ULG4ZHHSDZN4XA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.206213055 (106h13m50.90715795s) [noise=true , address=145.79.227.33:4898 , piece_id=XFD2TVHIKDC3NGPFNY4VLO4EDWE2VTSB3RT32HINFPSCAVEVZQRA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.224640360 (106h13m50.925585246s) [noise=true , address=145.79.227.21:4482 , piece_id=T246K4A3QSUUWBUL2AECHGA2SYJKEIVSMGZ2E7HPEL7FTN5N666A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.225634635 (106h13m50.92657953s) [noise=true , address=145.79.227.35:4501 , piece_id=HPFY3UD3P3UKMH3F352UTYVE4VYJQHJEZE4O6KMN4DA6H2H7VEMA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:27:24.232531017 (106h13m50.933475902s) [noise=true , address=145.79.227.44:4223 , piece_id=WSYN5Z3UPYBLQ72OTGVCL4BI5PJA6ZUIHMFWMTPZNYMOUVX3OSNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:13.947408711 (106h14m40.648353626s) [noise=true , address=45.91.116.7:4038 , piece_id=HBDQD2VYRPK5SLBDT54E3BEG3SUMFVKY66JVD6626PR5TBOX7RUQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:21.276856063 (106h14m47.977800958s) [noise=true , address=45.91.116.11:4137 , piece_id=UQEYQVYU5NQLEMQJV2U33DEOOCYYO6DNX4NV2ZSK2E5U3OEIWWGA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: <nil> Oct 31 21:28:24.069871428 (106h14m50.770816313s) [noise=true , address=77.90.31.15:4726 , piece_id=FJH2K5DUC6F6QZTU7BPDQTXKHBAFQ72SOCCAKCP6TCQO6G4FZUHQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12kMaukCkCJSc3ukAZu3thLgzXGGzNq44dGyEjCPqoY3PSYVEm3, address:77.90.31.15:4726): protocol: expected piece hash; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; write tcp 5.161.89.252:52754->77.90.31.15:4726: use of closed network connection; EOF Oct 31 21:28:24.069950206 (106h14m50.770895091s) [noise=true , address=77.90.31.11:4568 , piece_id=UEZNOBJV3B2UERQSCBUBNEIXDYR6Y73ENA3GYKXH4PYZJNIQNFHA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: read tcp 5.161.89.252:58260->77.90.31.11:4568: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.069974752 (106h14m50.770919647s) [noise=true , address=45.91.116.4:4947 , piece_id=BF5KBPTGYUL5TYX5BXUNGGS22FLVFNIVNVIQNRMKJIJICBYW64LA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1YTkM1w6HKKyw8z3eWomLMtJuFxsswqjZ5TmGyBeiZJE9QrFoB, address:45.91.116.4:4947): protocol: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31236->45.91.116.4:4947: use of closed network connection Oct 31 21:28:24.070022812 (106h14m50.770967707s) [noise=true , address=45.91.116.8:4076 , piece_id=DEIGDLGK7BBKHJRAQOCLOLKXHZLIDV7BB2K7DN6DYI2U7Q2C33IA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1WWDBuhCg17uYobyN1WCxT6TdHXjHg9NknseutbiCpKdGo1Nd1, address:45.91.116.8:4076): protocol: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:29812->45.91.116.8:4076: use of closed network connection Oct 31 21:28:24.070054221 (106h14m50.770999106s) [noise=true , address=77.90.31.13:4648 , piece_id=VMWDIWLJHK5KYGZY3EO6IE3DVN3W7VVAV5VAFDGREOG76QR5AWLA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15230->77.90.31.13:4648: use of closed network connection Oct 31 21:28:24.070113412 (106h14m50.771058297s) [noise=true , address=77.90.31.10:4546 , piece_id=OZQE54VEKSLC6AWZVREEH4ES6XFLBE6DI5C7QAP2Y6264AGXTPNQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK, address:77.90.31.10:4546): protocol: expected piece hash; context deadline exceeded; EOF Oct 31 21:28:24.070129752 (106h14m50.771074638s) [noise=true , address=77.90.31.12:4621 , piece_id=GT2ZFVCXDEMMP2ZXGHCDOY2IGNKTW47TV26WJOACMKPNN6OY5TXA, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12oSWAY52TJGVyZk4HTfifZBHQv42QsrAsk1Ks9PH6FoEmAZiN3, address:77.90.31.12:4621): protocol: read tcp 5.161.89.252:61442->77.90.31.12:4621: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:28:24.070184245 (106h14m50.77112914s) [noise=true , address=77.90.31.16:4737 , piece_id=R7454NFIMETFWXIWUJZAXRYVSZSM25UIQITZA6GNXDOXNZXDNY7A, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM, address:77.90.31.16:4737): protocol: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:16260->77.90.31.16:4737: use of closed network connection Oct 31 21:28:24.070274645 (106h14m50.77121954s) [noise=true , address=45.91.116.8:4054 , piece_id=ZDA72BIAED6RS2QWNVFNOQWEDXFX4USJ6HCELZ2ZS6DOOG5WBQWQ, seg_pos={0 0}, upload=93483]: Done uploading piece. err: ecclient: upload failed (node:1qC4ZTAobMwMUu7DQ34re4G4TV1jGEHg2UBXbaEM4k12EggNJW, address:45.91.116.8:4054): protocol: context deadline exceeded; context deadline exceeded Oct 31 21:28:24.070293379 (106h14m50.771238265s) [seg_pos={0 0}, upload=93483]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Oct 31 21:28:24.087202136 (106h14m50.788147031s) [upload=93483]: Done uploading segments... Oct 31 21:28:24.087224858 (106h14m50.788169744s) [upload=93483]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 959.061242ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...X.X...............................X......................] duration: 1.820821052s FAIL (2025-10-31 22:11:18 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.820919868s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.....X.................................X....................] duration: 609.597954ms FAIL (2025-10-31 22:10:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 609.617452ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.............................X.......................] duration: 411.655591ms FAIL (2025-10-31 22:10:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 411.675828ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.................................X.......................] duration: 406.362145ms FAIL (2025-10-31 22:10:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 21:53:55.636571581 (106h40m22.337516497s) [upload=93876]: Starting upload Oct 31 21:53:55.636723827 (106h40m22.337668742s) [upload=93876]: Done starting upload Oct 31 21:53:55.636846417 (106h40m22.337791352s) [upload=93876]: Uploading segments... Oct 31 21:53:55.637224375 (106h40m22.33816931s) [upload=93876]: Got next segment. Inline: false Oct 31 21:53:55.637325275 (106h40m22.3382702s) [upload=93876]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 21:53:55.637334242 (106h40m22.338279156s) [upload=93876]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 21:53:55.657334926 (106h40m22.358279841s) [seg_pos={0 0}, upload=93876]: Begin upload segment... Oct 31 21:53:55.657376123 (106h40m22.358321039s) [seg_pos={0 0}, upload=93876]: Done begin upload segment. Oct 31 21:53:55.657404807 (106h40m22.358349723s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.657436727 (106h40m22.358381642s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658171756 (106h40m22.35911667s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658246776 (106h40m22.359191691s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658568789 (106h40m22.359513705s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.658921311 (106h40m22.359866225s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659015637 (106h40m22.359960553s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659398385 (106h40m22.36034331s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659537967 (106h40m22.360482892s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.659905977 (106h40m22.360850893s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660336394 (106h40m22.36128131s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660367613 (106h40m22.361312528s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.660772091 (106h40m22.361717007s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661090197 (106h40m22.362035113s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661165078 (106h40m22.362109993s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.661754223 (106h40m22.362699138s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Uploading piece... Oct 31 21:53:55.749091443 (106h40m22.450036338s) [upload=93876]: Next returned nil segment Oct 31 21:53:55.749134874 (106h40m22.450079769s) [upload=93876]: Waiting for error group managing segments... Oct 31 21:53:55.758087501 (106h40m22.459032385s) [noise=true , address=145.79.227.18:4005 , piece_id=HUC3W2CMGFTS6ZMSXYO4KKPWAIXCIKAWYPDIVONYGLUS6YVJUXHA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.758740425 (106h40m22.459685301s) [noise=true , address=145.79.227.17:4970 , piece_id=DBPOIR3D4LPW5JE6AXTI7IFOCM7PF2XKPGIUGC4X5RWWJOWH6R4Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759108115 (106h40m22.46005299s) [noise=true , address=145.79.227.11:4729 , piece_id=PLZHKJ7OR6NK6CT4R3FLZWPGF25UIBY2C2IWVJRCDYKW23WLJI2A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759663576 (106h40m22.460608451s) [noise=true , address=145.79.227.27:4690 , piece_id=SGUSU5KRFKILSJVY3K62GTJVJMNIFVPWYMNJWJNWC5VD3KHSF5GA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.759779894 (106h40m22.460724779s) [noise=true , address=145.79.227.25:4618 , piece_id=FCL3VVBGZBH64KLA4XFLBKHPIR5AE4H2IZFKKSTELTLXTO3LJMQQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760041835 (106h40m22.46098672s) [noise=true , address=145.79.227.32:4835 , piece_id=NXMRRTHQUAJTFD6ZYOFLXOJL3CNIG5FW5OVNNBQZ6ATOJGHLX5SQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760200122 (106h40m22.461145007s) [noise=true , address=145.79.227.15:4874 , piece_id=G5ASGJ4T5J6RGYJAFR7CMCMYGUG6AWZJTEZVO3KLKK76CNL6P75Q, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760415316 (106h40m22.461360201s) [noise=true , address=145.79.227.21:4475 , piece_id=MJATSGZ5ICSAPNAYEKZLEXK56AG6HTINRM2WMYE3CJEJKLQYT7XA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760602286 (106h40m22.461547171s) [noise=true , address=145.79.227.27:4691 , piece_id=GF45MMIGRCDYQSDTWZHYER5VOHVKCGHEXDDOBMMEDMMI24C335JA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.760776313 (106h40m22.461721198s) [noise=true , address=145.79.227.31:4799 , piece_id=Y5CZ4LLK2NBIDXPWDAWEQZBRKWDKIRPCU72T25HWSD5BQW53Z2RQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.761557137 (106h40m22.462502022s) [noise=true , address=145.79.227.21:4474 , piece_id=KFV4SPNC47UGX5X5JRSLWA4ZC5Q5VI7ROZ4YFO7JMVWCWETDWDXQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:53:55.776736230 (106h40m22.477681114s) [noise=true , address=145.79.227.10:4402 , piece_id=Z646NZ7OWMIO6J2HVNJKNNXOGDYPHG7GHUUMGM3W54UNXITOQQHQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: <nil> Oct 31 21:54:25.625986871 (106h40m52.326931776s) [noise=true , address=45.91.116.14:4190 , piece_id=D5BI7QM7LCOK2RAEMFUANO56ZOGJVFQDJPZTKXYKVO7JROH3CGVA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:47216->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626009954 (106h40m52.326954839s) [noise=true , address=77.90.31.13:4626 , piece_id=5DCPIYCL5BB3DAANE3XGJJSSVNFUF5EV7TEIK5HJYU4G4LZRNBOQ, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: read tcp 5.161.89.252:12918->77.90.31.13:4626: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626144627 (106h40m52.327089522s) [noise=true , address=77.90.31.18:4805 , piece_id=Q4RMSFCWFI67O6GGYRB5SEDACQUH3XEUPNJQUVTC2EI65NQ5TH3A, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:46724->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626160907 (106h40m52.327105803s) [noise=true , address=77.90.31.16:4734 , piece_id=7D44KYJPUMILGVSDXUX6L36YZKQIBXGYWLMZXJZ74HNVB6QQRVSA, seg_pos={0 0}, upload=93876]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:16936->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 21:54:25.626188069 (106h40m52.327132953s) [seg_pos={0 0}, upload=93876]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 21:54:25.643440038 (106h40m52.344384934s) [upload=93876]: Done uploading segments... Oct 31 21:54:25.643452923 (106h40m52.344397818s) [upload=93876]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 406.383164ms core uplink us-select-1

1
...
469
470
471
472
473
...
2340