Orbiter History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.024053513s FAIL (2025-09-22 03:15:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.024076417s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 734.783766ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:15:50 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 734.805887ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 397.975793ms FAIL (2025-09-22 03:15:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 397.996402ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 4.866963944s FAIL (2025-09-22 03:15:38 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.866980537s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 815.392012ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:15:24 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 815.411989ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [XX..X....XX.XXXX.XX..X...X........X.........................] duration: 536.711247ms FAIL (2025-09-22 03:15:18 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 536.735983ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [................................X.......XX..................] duration: 2.163450609s FAIL (2025-09-22 03:15:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.163469155s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 374.986331ms FAIL (2025-09-22 03:15:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 375.01808ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X.XX..XX..X.........X...X.......XX..........XX.....X..X...] duration: 399.213754ms FAIL (2025-09-22 03:14:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 399.233682ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.XX..X....XX.XXXX.XX..X...X........X........................] duration: 442.018104ms FAIL (2025-09-22 03:14:56 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 442.039634ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 4.943279476s FAIL (2025-09-22 03:14:53 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.943296418s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...X.XX..XX..X.........X...X.......XX..........XX.....X..X..] duration: 385.606658ms FAIL (2025-09-22 03:14:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 385.626264ms edge gateway us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 738.097212ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:14:33 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 738.115105ms core satellite ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................................X.......XX.................] duration: 2.332513434s FAIL (2025-09-22 03:14:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.332535515s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..XX..X....XX.XXXX.XX..X...X........X.......................] duration: 457.352414ms FAIL (2025-09-22 03:14:22 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 457.378112ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.090254444s FAIL (2025-09-22 03:14:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.090282506s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 391.676839ms FAIL (2025-09-22 03:13:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 391.697468ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................................X.......XX................] duration: 2.162617576s FAIL (2025-09-22 03:13:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.162656639s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....X.XX..XX..X.........X...X.......XX..........XX.....X..X.] duration: 373.022816ms FAIL (2025-09-22 03:13:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 373.033686ms edge gateway us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 736.136933ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:13:28 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 736.159676ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...XX..X....XX.XXXX.XX..X...X........X......................] duration: 492.667135ms FAIL (2025-09-22 03:13:24 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 492.688033ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....XX..X....XX.XXXX.XX..X...X........X.....................] duration: 723.517623ms FAIL (2025-09-22 03:13:09 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 723.54849ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 420.826219ms FAIL (2025-09-22 03:13:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 420.847861ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.032376348s FAIL (2025-09-22 03:13:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.032391837s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....XX..X....XX.XXXX.XX..X...X........X....................] duration: 436.756169ms FAIL (2025-09-22 03:12:51 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 436.777809ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 415.84652ms FAIL (2025-09-22 03:12:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 415.870436ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................................X.......XX...............] duration: 2.178379025s FAIL (2025-09-22 03:12:50 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.178423928s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 4.954827074s FAIL (2025-09-22 03:12:49 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.954852492s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.....X.XX..XX..X.........X...X.......XX..........XX.....X..] duration: 410.343142ms FAIL (2025-09-22 03:12:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 410.361676ms edge gateway us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 822.270738ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:12:36 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 822.281939ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 413.741239ms FAIL (2025-09-22 03:12:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 413.755005ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......XX..X....XX.XXXX.XX..X...X........X...................] duration: 460.550261ms FAIL (2025-09-22 03:12:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 460.569447ms core uplink us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 738.340295ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:12:02 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 738.361936ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 734.9ms FAIL (2025-09-22 03:11:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 734.917182ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.153058515s FAIL (2025-09-22 03:11:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.153081928s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X.....X.XX..XX..X.........X...X.......XX..........XX.....X.] duration: 416.196386ms FAIL (2025-09-22 03:11:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 416.215934ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................................X.......XX..............] duration: 2.210042474s FAIL (2025-09-22 03:11:49 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.210071598s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X.....X.XX..XX..X.........X...X.......XX..........XX.....X] duration: 30.000627368s FAIL (2025-09-22 03:11:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000643848s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [X......XX..X....XX.XXXX.XX..X...X........X..................] duration: 565.310732ms FAIL (2025-09-22 03:11:14 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 565.334867ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 475.452816ms FAIL (2025-09-22 03:11:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 475.491037ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [XX......XX..X....XX.XXXX.XX..X...X........X.................] duration: 455.851544ms FAIL (2025-09-22 03:10:57 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 02:57:08.281940234 (564h48m43.634725342s) [upload=497064]: Starting upload Sep 22 02:57:08.281995938 (564h48m43.634781046s) [upload=497064]: Done starting upload Sep 22 02:57:08.282071449 (564h48m43.634856578s) [upload=497064]: Uploading segments... Sep 22 02:57:08.282236871 (564h48m43.635021979s) [upload=497064]: Got next segment. Inline: false Sep 22 02:57:08.282274551 (564h48m43.63505968s) [upload=497064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:08.282281825 (564h48m43.635066953s) [upload=497064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:08.301081935 (564h48m43.653867063s) [seg_pos={0 0}, upload=497064]: Begin upload segment... Sep 22 02:57:08.301119636 (564h48m43.653904764s) [seg_pos={0 0}, upload=497064]: Done begin upload segment. Sep 22 02:57:08.301152938 (564h48m43.653938046s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301171593 (564h48m43.653956721s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301475353 (564h48m43.654260461s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301658757 (564h48m43.654443885s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301723969 (564h48m43.654509077s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.301945896 (564h48m43.654730995s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302047447 (564h48m43.654832575s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302173934 (564h48m43.654959042s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302400249 (564h48m43.655185346s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302498714 (564h48m43.655283851s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302619369 (564h48m43.655404477s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.302934452 (564h48m43.65571958s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303299276 (564h48m43.656084414s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.303783665 (564h48m43.656568794s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304260449 (564h48m43.657045578s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.304649841 (564h48m43.657434969s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Uploading piece... Sep 22 02:57:08.408885477 (564h48m43.761670574s) [upload=497064]: Next returned nil segment Sep 22 02:57:08.408896908 (564h48m43.761682006s) [upload=497064]: Waiting for error group managing segments... Sep 22 02:57:08.417535079 (564h48m43.770320177s) [noise=true , address=45.91.116.14:4189 , piece_id=HTW6U45MRHEXKNR35DUY374KMNVKPNHRJV4K5TOXW2EI53COT5PQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.417865899 (564h48m43.770650998s) [noise=true , address=45.91.116.12:4225 , piece_id=3UHKLHTRTFTOPDDM5QGGKQYO6H3KTSPNAYUV2YBZGGWBOPVLEUUQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418178976 (564h48m43.770964105s) [noise=true , address=45.91.116.12:4227 , piece_id=RR3PDGKUTP3WCP3SYB7WRYEYPTMCRXIGLVFXMX5GMTPB6JNRCVZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418365918 (564h48m43.771151057s) [noise=true , address=45.91.116.9:4262 , piece_id=BHF7N2IOY4QEXS2JGNJMQ2MSLTVFTXROPLLST62HSJKPECW6QH5Q, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418472909 (564h48m43.771258007s) [noise=true , address=77.90.31.18:4806 , piece_id=Z67HG7K7CSZCUMKR2UFWMDKM75DAXJ3P62ZZKMWEO4PM2G3IBDZQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.418832323 (564h48m43.771617421s) [noise=true , address=45.91.116.8:4047 , piece_id=56WTV2CNWFYGX2QHT25LCPHMQH32VZEGETFW2YRJVR2VZVI6IDHQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419246100 (564h48m43.772031198s) [noise=true , address=45.91.116.11:4119 , piece_id=CQT53UNDD6GB3BRQPMPGXO36A2C3GUWHTWIBEYYHERLW3GOI55VQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419414676 (564h48m43.772199814s) [noise=true , address=45.91.116.6:4974 , piece_id=MKHRQQDRD5BRKLWU6VCERUF2ZMEGXK7ORBPGXZJJD3GRQ5JESPBQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.419533740 (564h48m43.772318837s) [noise=true , address=45.91.116.4:4937 , piece_id=UATS2IK3YMOGIM3AIX6LC32REBX654P7SE6CGVLQFPIBGCCEKPMQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420666545 (564h48m43.773451683s) [noise=true , address=45.91.116.13:4155 , piece_id=ABEANYFL6VZQD56OEL7XFLBWO7SXV6SC2ICBBDGPTSNGAWANBDRA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.420852824 (564h48m43.773637923s) [noise=true , address=77.90.31.19:4842 , piece_id=URJOWGORIFGZDMH2OLKUQWAFBJKYD54HHZ2S6J3RENWI3LZHGGDQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:08.421253537 (564h48m43.774038665s) [noise=true , address=77.90.31.13:4627 , piece_id=U6N7WUE52AKOQJGQG2XPMOQQSWOPKJP4A62D6GEUDVUTFODDKMZA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: <nil> Sep 22 02:57:38.269181747 (564h49m13.621966875s) [noise=true , address=145.79.227.11:4730 , piece_id=NIT6NGSYDU5NTHIZTZPUUV5YOF6Z2T3EWS6PEWQYDGIMJWYA5RMA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269239155 (564h49m13.622024283s) [noise=true , address=145.79.227.19:4077 , piece_id=N52U3LQ2N2YCPUGJRRXEYELXMQQ6F463WGK6HK2UJM6C6MFNCPFA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269287445 (564h49m13.622072573s) [noise=true , address=145.79.227.14:4838 , piece_id=HUEUHZ4O64WZ255NBVPCQKK6SWHZQMLXW2BP2AB7CZNTCJ55JLGQ, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:57:38.269309677 (564h49m13.622094805s) [noise=true , address=145.79.227.17:4970 , piece_id=G2NWG5IUI2PXNQGUZ5TGRGQ6TIOOJA75C7XDZU3PLPNC6CMVBCBA, seg_pos={0 0}, upload=497064]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: rpc: dial tcp 145.79.227.17:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:57:38.269333642 (564h49m13.62211878s) [seg_pos={0 0}, upload=497064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 02:57:38.288426071 (564h49m13.641211209s) [upload=497064]: Done uploading segments... Sep 22 02:57:38.288464884 (564h49m13.641250012s) [upload=497064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 455.876269ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 4.915321272s FAIL (2025-09-22 03:10:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.915348403s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 754.067368ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:10:53 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 754.0945ms core satellite ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X.......XX.............] duration: 2.326475833s FAIL (2025-09-22 03:10:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.326504286s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 353.272606ms FAIL (2025-09-22 03:10:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 353.290187ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X.....X.XX..XX..X.........X...X.......XX..........XX.....] duration: 489.84244ms FAIL (2025-09-22 03:10:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 489.857257ms edge gateway us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 750.146099ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:10:13 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 750.167199ms core satellite ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................................X.......XX............] duration: 2.233235823s FAIL (2025-09-22 03:10:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.233268894s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 452.430382ms FAIL (2025-09-22 03:10:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 452.453555ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 4.927437485s FAIL (2025-09-22 03:10:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.927497629s core uplink ap1

Orbiter Check History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)