Orbiter History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........X.......XX.........................................] duration: 2.181541249s FAIL (2025-09-22 03:35:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.181573589s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 348.492121ms FAIL (2025-09-22 03:34:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 348.515155ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 747.573111ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:34:48 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 747.593399ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X........X....................................X..X...X...] duration: 401.57259ms FAIL (2025-09-22 03:34:35 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:32:07.812453293 (565h23m43.165238421s) [upload=497588]: Starting upload Sep 22 03:32:07.812536730 (565h23m43.165321857s) [upload=497588]: Done starting upload Sep 22 03:32:07.812614876 (565h23m43.165400004s) [upload=497588]: Uploading segments... Sep 22 03:32:07.813014276 (565h23m43.165799405s) [upload=497588]: Got next segment. Inline: false Sep 22 03:32:07.813065041 (565h23m43.165850169s) [upload=497588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:32:07.813071283 (565h23m43.165856411s) [upload=497588]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:32:07.832264650 (565h23m43.185049779s) [seg_pos={0 0}, upload=497588]: Begin upload segment... Sep 22 03:32:07.832325865 (565h23m43.185111004s) [seg_pos={0 0}, upload=497588]: Done begin upload segment. Sep 22 03:32:07.832357404 (565h23m43.185142543s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832377582 (565h23m43.18516273s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832857704 (565h23m43.185642831s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833012814 (565h23m43.185797953s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833229661 (565h23m43.186014799s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833582865 (565h23m43.186367992s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833611729 (565h23m43.186396857s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833959932 (565h23m43.18674506s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834187939 (565h23m43.186973078s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834324006 (565h23m43.187109134s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834734065 (565h23m43.187519203s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835289818 (565h23m43.188074946s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835852073 (565h23m43.188637201s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836431540 (565h23m43.189216669s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836920277 (565h23m43.189705405s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.837330798 (565h23m43.190115926s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.946789796 (565h23m43.299574925s) [upload=497588]: Next returned nil segment Sep 22 03:32:07.946807450 (565h23m43.299592577s) [upload=497588]: Waiting for error group managing segments... Sep 22 03:32:07.954965559 (565h23m43.307750687s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.955795567 (565h23m43.308580705s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956015469 (565h23m43.308800597s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956427853 (565h23m43.309212982s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956805682 (565h23m43.30959081s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957118920 (565h23m43.309904048s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957580135 (565h23m43.310365264s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957702986 (565h23m43.310488124s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958396598 (565h23m43.311181726s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958626870 (565h23m43.311411998s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958949996 (565h23m43.311735134s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.959310573 (565h23m43.312095701s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:37.798995278 (565h24m13.151780386s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799032588 (565h24m13.151817696s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:32:37.799071170 (565h24m13.151856268s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799084204 (565h24m13.151869303s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799099984 (565h24m13.151885083s) [seg_pos={0 0}, upload=497588]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:32:37.812188355 (565h24m13.164973463s) [upload=497588]: Done uploading segments... Sep 22 03:32:37.812200288 (565h24m13.164985396s) [upload=497588]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 401.602526ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 414.242185ms FAIL (2025-09-22 03:34:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 414.271129ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [X.......XX..........XX.....X..X...X..........X...X.XX...XXX.] duration: 415.136408ms FAIL (2025-09-22 03:34:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 415.154172ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 4.973402848s FAIL (2025-09-22 03:34:12 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.973426132s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X.......XX........................................] duration: 2.324606398s FAIL (2025-09-22 03:34:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.324625905s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 745.184532ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:34:03 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 745.209969ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X...X........X....................................X..X...X..] duration: 419.417397ms FAIL (2025-09-22 03:34:00 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:32:07.812453293 (565h23m43.165238421s) [upload=497588]: Starting upload Sep 22 03:32:07.812536730 (565h23m43.165321857s) [upload=497588]: Done starting upload Sep 22 03:32:07.812614876 (565h23m43.165400004s) [upload=497588]: Uploading segments... Sep 22 03:32:07.813014276 (565h23m43.165799405s) [upload=497588]: Got next segment. Inline: false Sep 22 03:32:07.813065041 (565h23m43.165850169s) [upload=497588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:32:07.813071283 (565h23m43.165856411s) [upload=497588]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:32:07.832264650 (565h23m43.185049779s) [seg_pos={0 0}, upload=497588]: Begin upload segment... Sep 22 03:32:07.832325865 (565h23m43.185111004s) [seg_pos={0 0}, upload=497588]: Done begin upload segment. Sep 22 03:32:07.832357404 (565h23m43.185142543s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832377582 (565h23m43.18516273s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832857704 (565h23m43.185642831s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833012814 (565h23m43.185797953s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833229661 (565h23m43.186014799s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833582865 (565h23m43.186367992s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833611729 (565h23m43.186396857s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833959932 (565h23m43.18674506s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834187939 (565h23m43.186973078s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834324006 (565h23m43.187109134s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834734065 (565h23m43.187519203s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835289818 (565h23m43.188074946s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835852073 (565h23m43.188637201s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836431540 (565h23m43.189216669s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836920277 (565h23m43.189705405s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.837330798 (565h23m43.190115926s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.946789796 (565h23m43.299574925s) [upload=497588]: Next returned nil segment Sep 22 03:32:07.946807450 (565h23m43.299592577s) [upload=497588]: Waiting for error group managing segments... Sep 22 03:32:07.954965559 (565h23m43.307750687s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.955795567 (565h23m43.308580705s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956015469 (565h23m43.308800597s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956427853 (565h23m43.309212982s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956805682 (565h23m43.30959081s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957118920 (565h23m43.309904048s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957580135 (565h23m43.310365264s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957702986 (565h23m43.310488124s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958396598 (565h23m43.311181726s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958626870 (565h23m43.311411998s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958949996 (565h23m43.311735134s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.959310573 (565h23m43.312095701s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:37.798995278 (565h24m13.151780386s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799032588 (565h24m13.151817696s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:32:37.799071170 (565h24m13.151856268s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799084204 (565h24m13.151869303s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799099984 (565h24m13.151885083s) [seg_pos={0 0}, upload=497588]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:32:37.812188355 (565h24m13.164973463s) [upload=497588]: Done uploading segments... Sep 22 03:32:37.812200288 (565h24m13.164985396s) [upload=497588]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 419.446151ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.X.......XX..........XX.....X..X...X..........X...X.XX...XXX] duration: 30.00129927s FAIL (2025-09-22 03:33:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001321983s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 434.695026ms FAIL (2025-09-22 03:33:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 434.727097ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.009927428s FAIL (2025-09-22 03:33:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.009952815s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X...X........X....................................X..X...X.] duration: 464.477615ms FAIL (2025-09-22 03:33:17 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:32:07.812453293 (565h23m43.165238421s) [upload=497588]: Starting upload Sep 22 03:32:07.812536730 (565h23m43.165321857s) [upload=497588]: Done starting upload Sep 22 03:32:07.812614876 (565h23m43.165400004s) [upload=497588]: Uploading segments... Sep 22 03:32:07.813014276 (565h23m43.165799405s) [upload=497588]: Got next segment. Inline: false Sep 22 03:32:07.813065041 (565h23m43.165850169s) [upload=497588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:32:07.813071283 (565h23m43.165856411s) [upload=497588]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:32:07.832264650 (565h23m43.185049779s) [seg_pos={0 0}, upload=497588]: Begin upload segment... Sep 22 03:32:07.832325865 (565h23m43.185111004s) [seg_pos={0 0}, upload=497588]: Done begin upload segment. Sep 22 03:32:07.832357404 (565h23m43.185142543s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832377582 (565h23m43.18516273s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832857704 (565h23m43.185642831s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833012814 (565h23m43.185797953s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833229661 (565h23m43.186014799s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833582865 (565h23m43.186367992s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833611729 (565h23m43.186396857s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833959932 (565h23m43.18674506s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834187939 (565h23m43.186973078s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834324006 (565h23m43.187109134s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834734065 (565h23m43.187519203s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835289818 (565h23m43.188074946s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835852073 (565h23m43.188637201s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836431540 (565h23m43.189216669s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836920277 (565h23m43.189705405s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.837330798 (565h23m43.190115926s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.946789796 (565h23m43.299574925s) [upload=497588]: Next returned nil segment Sep 22 03:32:07.946807450 (565h23m43.299592577s) [upload=497588]: Waiting for error group managing segments... Sep 22 03:32:07.954965559 (565h23m43.307750687s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.955795567 (565h23m43.308580705s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956015469 (565h23m43.308800597s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956427853 (565h23m43.309212982s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956805682 (565h23m43.30959081s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957118920 (565h23m43.309904048s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957580135 (565h23m43.310365264s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957702986 (565h23m43.310488124s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958396598 (565h23m43.311181726s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958626870 (565h23m43.311411998s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958949996 (565h23m43.311735134s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.959310573 (565h23m43.312095701s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:37.798995278 (565h24m13.151780386s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799032588 (565h24m13.151817696s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:32:37.799071170 (565h24m13.151856268s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799084204 (565h24m13.151869303s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799099984 (565h24m13.151885083s) [seg_pos={0 0}, upload=497588]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:32:37.812188355 (565h24m13.164973463s) [upload=497588]: Done uploading segments... Sep 22 03:32:37.812200288 (565h24m13.164985396s) [upload=497588]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 464.500068ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X.......XX.......................................] duration: 2.46867321s FAIL (2025-09-22 03:33:08 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.468701374s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X.......XX..........XX.....X..X...X..........X...X.XX...XX] duration: 30.001448237s FAIL (2025-09-22 03:33:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001472353s edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 4.981173291s FAIL (2025-09-22 03:33:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.981201354s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 436.432507ms FAIL (2025-09-22 03:33:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 436.462382ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 759.493904ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:32:58 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 759.517669ms core satellite ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.06923953s FAIL (2025-09-22 03:32:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.069260961s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X...X........X....................................X..X...X] duration: 30.014311838s FAIL (2025-09-22 03:32:37 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:32:07.812453293 (565h23m43.165238421s) [upload=497588]: Starting upload Sep 22 03:32:07.812536730 (565h23m43.165321857s) [upload=497588]: Done starting upload Sep 22 03:32:07.812614876 (565h23m43.165400004s) [upload=497588]: Uploading segments... Sep 22 03:32:07.813014276 (565h23m43.165799405s) [upload=497588]: Got next segment. Inline: false Sep 22 03:32:07.813065041 (565h23m43.165850169s) [upload=497588]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:32:07.813071283 (565h23m43.165856411s) [upload=497588]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:32:07.832264650 (565h23m43.185049779s) [seg_pos={0 0}, upload=497588]: Begin upload segment... Sep 22 03:32:07.832325865 (565h23m43.185111004s) [seg_pos={0 0}, upload=497588]: Done begin upload segment. Sep 22 03:32:07.832357404 (565h23m43.185142543s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832377582 (565h23m43.18516273s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.832857704 (565h23m43.185642831s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833012814 (565h23m43.185797953s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833229661 (565h23m43.186014799s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833582865 (565h23m43.186367992s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833611729 (565h23m43.186396857s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.833959932 (565h23m43.18674506s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834187939 (565h23m43.186973078s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834324006 (565h23m43.187109134s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.834734065 (565h23m43.187519203s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835289818 (565h23m43.188074946s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.835852073 (565h23m43.188637201s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836431540 (565h23m43.189216669s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.836920277 (565h23m43.189705405s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.837330798 (565h23m43.190115926s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Uploading piece... Sep 22 03:32:07.946789796 (565h23m43.299574925s) [upload=497588]: Next returned nil segment Sep 22 03:32:07.946807450 (565h23m43.299592577s) [upload=497588]: Waiting for error group managing segments... Sep 22 03:32:07.954965559 (565h23m43.307750687s) [noise=true , address=45.91.116.6:4973 , piece_id=GZMMSPHKVW4I34E5YXIVC4YNSN6USCPRTBR6SBVZNTDU3UTSWH5A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.955795567 (565h23m43.308580705s) [noise=true , address=45.91.116.5:4297 , piece_id=GWNJ7BZKUA4QOLFB3JLFAFQC7YBO4EZBGNOV4HNWJ2SWBKKI6B6A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956015469 (565h23m43.308800597s) [noise=true , address=77.90.31.10:4518 , piece_id=UFSTU4PTB6LXUHC6CZBK4ZLY5XXTHM2TPGZAJ4IHSTTITPLJFYNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956427853 (565h23m43.309212982s) [noise=true , address=45.91.116.8:4048 , piece_id=7IL55RVO7TT2KEF2HVO7JCNY3CYQWCUTX7BY27J4TWGYC7AWGKNA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.956805682 (565h23m43.30959081s) [noise=true , address=45.91.116.4:4940 , piece_id=2VZCBORQXQDXQTWVQ6W3J3K35JTGSCJQJCXTYJFE7WPIRIR2FR7Q, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957118920 (565h23m43.309904048s) [noise=true , address=77.90.31.17:4772 , piece_id=2IIJFLACSWXIMVL4AJNI6HRBFT5QHXTJVVQNI66HVINXAH6JTTFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957580135 (565h23m43.310365264s) [noise=true , address=45.91.116.9:4261 , piece_id=2OPNYTS55Y2L63NVRAG4MCC3SUW64LISVDI6GFTP43SJADR3GKWA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.957702986 (565h23m43.310488124s) [noise=true , address=77.90.31.11:4553 , piece_id=4DMJ5TVVY6PWT5HLLF2GFRUWSBYTVTJNNTGZ7MEO3OEWCJVXXZFQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958396598 (565h23m43.311181726s) [noise=true , address=45.91.116.13:4154 , piece_id=TF43SOUKYROLKHGVQF4ETIO4LVZ6B6A2PUOESUV7VULXV3KBG77A, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958626870 (565h23m43.311411998s) [noise=true , address=77.90.31.12:4592 , piece_id=F55ZDNPLEF3IYA6NSKQHMVVDA7YPVR3EDM37TINQILHBN2ZECIAQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.958949996 (565h23m43.311735134s) [noise=true , address=45.91.116.14:4191 , piece_id=BHU65TECZN2DCHCHB4X3N52CBVAL7QHXNDA7627MPH3U5UVTYVZA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:07.959310573 (565h23m43.312095701s) [noise=true , address=77.90.31.15:4697 , piece_id=GIT4LCOSMBRGFAQCGFEXODVRJ6DLXNZBYMXVM7RGE7FVD5VE47TA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: <nil> Sep 22 03:32:37.798995278 (565h24m13.151780386s) [noise=true , address=145.79.227.19:4079 , piece_id=L3ZDJR7AJAO7NQLWRZB25TZKRMPB7RTW55CPQNVZADL35MGGSUCA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799032588 (565h24m13.151817696s) [noise=true , address=145.79.227.12:4767 , piece_id=D3B4NGA5EXGP4DRP6NCZBDB2PPGI67P4LJFS7LCDSHYVWPNVFTXA, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:32:37.799071170 (565h24m13.151856268s) [noise=true , address=145.79.227.6:4257 , piece_id=33Y42RUGVAAHIB54DEJPUDB3KGFI4M6IC4XWDQL6FB5P4LS7MNYQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799084204 (565h24m13.151869303s) [noise=true , address=145.79.227.2:4114 , piece_id=KYJWRFRFHAGVQEHRFMXZJ6U6AM2XR5FTDUPOLGQBZVIPSAQHNHSQ, seg_pos={0 0}, upload=497588]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:32:37.799099984 (565h24m13.151885083s) [seg_pos={0 0}, upload=497588]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:32:37.812188355 (565h24m13.164973463s) [upload=497588]: Done uploading segments... Sep 22 03:32:37.812200288 (565h24m13.164985396s) [upload=497588]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.014332727s core uplink us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 813.947541ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:32:36 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 813.965234ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 484.246404ms FAIL (2025-09-22 03:32:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 484.269527ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.058922627s FAIL (2025-09-22 03:32:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.058944439s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X.......XX..........XX.....X..X...X..........X...X.XX...X] duration: 30.000560901s FAIL (2025-09-22 03:32:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000578554s edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X.......XX......................................] duration: 2.206698858s FAIL (2025-09-22 03:32:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.206724495s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 776.038502ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:31:53 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 776.06381ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 385.864483ms FAIL (2025-09-22 03:31:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 385.891173ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 730.400668ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:31:35 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 730.418953ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..X...X........X....................................X..X...] duration: 538.744166ms FAIL (2025-09-22 03:31:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:28:30.304830750 (565h20m5.657615848s) [upload=497533]: Starting upload Sep 22 03:28:30.304903216 (565h20m5.657688324s) [upload=497533]: Done starting upload Sep 22 03:28:30.304993495 (565h20m5.657778603s) [upload=497533]: Uploading segments... Sep 22 03:28:30.305151993 (565h20m5.657937101s) [upload=497533]: Got next segment. Inline: false Sep 22 03:28:30.305194232 (565h20m5.65797934s) [upload=497533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:28:30.305198770 (565h20m5.657983878s) [upload=497533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:28:30.328856497 (565h20m5.681641605s) [seg_pos={0 0}, upload=497533]: Begin upload segment... Sep 22 03:28:30.328913333 (565h20m5.681698442s) [seg_pos={0 0}, upload=497533]: Done begin upload segment. Sep 22 03:28:30.328950693 (565h20m5.681735801s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.328961063 (565h20m5.681746161s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329079445 (565h20m5.681864553s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329441154 (565h20m5.682226262s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329457795 (565h20m5.682242903s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329862725 (565h20m5.682647833s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329882192 (565h20m5.68266729s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330250232 (565h20m5.68303534s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330290668 (565h20m5.683075776s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330667806 (565h20m5.683452934s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330673116 (565h20m5.683458224s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331075671 (565h20m5.683860779s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331437830 (565h20m5.684222938s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331789951 (565h20m5.68457507s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.332160136 (565h20m5.684945244s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.337945593 (565h20m5.690730701s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.421652625 (565h20m5.774437724s) [upload=497533]: Next returned nil segment Sep 22 03:28:30.421674036 (565h20m5.774459134s) [upload=497533]: Waiting for error group managing segments... Sep 22 03:28:30.430729979 (565h20m5.783515077s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432230315 (565h20m5.785015413s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432851100 (565h20m5.785636198s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432864114 (565h20m5.785649212s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433283611 (565h20m5.78606871s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433312286 (565h20m5.786097384s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433608802 (565h20m5.7863939s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433900248 (565h20m5.786685346s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434889345 (565h20m5.787674443s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434926946 (565h20m5.787712043s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.435204566 (565h20m5.787989665s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.436237565 (565h20m5.789022653s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:29:00.291216686 (565h20m35.644001784s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291236933 (565h20m35.644022042s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291272671 (565h20m35.644057769s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291291958 (565h20m35.644077066s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291307757 (565h20m35.644092865s) [seg_pos={0 0}, upload=497533]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:29:00.309425256 (565h20m35.662210354s) [upload=497533]: Done uploading segments... Sep 22 03:29:00.309440985 (565h20m35.662226093s) [upload=497533]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 538.768702ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.344213854s FAIL (2025-09-22 03:31:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.344236848s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X.......XX.....................................] duration: 2.164025929s FAIL (2025-09-22 03:31:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.16404717s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 434.877588ms FAIL (2025-09-22 03:30:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 434.899189ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X.......XX....................................] duration: 2.255309446s FAIL (2025-09-22 03:30:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.255332029s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 753.756046ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:30:40 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 753.780181ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX..X...X........X....................................X..X..] duration: 877.455839ms FAIL (2025-09-22 03:30:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:28:30.304830750 (565h20m5.657615848s) [upload=497533]: Starting upload Sep 22 03:28:30.304903216 (565h20m5.657688324s) [upload=497533]: Done starting upload Sep 22 03:28:30.304993495 (565h20m5.657778603s) [upload=497533]: Uploading segments... Sep 22 03:28:30.305151993 (565h20m5.657937101s) [upload=497533]: Got next segment. Inline: false Sep 22 03:28:30.305194232 (565h20m5.65797934s) [upload=497533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:28:30.305198770 (565h20m5.657983878s) [upload=497533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:28:30.328856497 (565h20m5.681641605s) [seg_pos={0 0}, upload=497533]: Begin upload segment... Sep 22 03:28:30.328913333 (565h20m5.681698442s) [seg_pos={0 0}, upload=497533]: Done begin upload segment. Sep 22 03:28:30.328950693 (565h20m5.681735801s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.328961063 (565h20m5.681746161s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329079445 (565h20m5.681864553s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329441154 (565h20m5.682226262s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329457795 (565h20m5.682242903s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329862725 (565h20m5.682647833s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329882192 (565h20m5.68266729s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330250232 (565h20m5.68303534s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330290668 (565h20m5.683075776s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330667806 (565h20m5.683452934s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330673116 (565h20m5.683458224s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331075671 (565h20m5.683860779s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331437830 (565h20m5.684222938s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331789951 (565h20m5.68457507s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.332160136 (565h20m5.684945244s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.337945593 (565h20m5.690730701s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.421652625 (565h20m5.774437724s) [upload=497533]: Next returned nil segment Sep 22 03:28:30.421674036 (565h20m5.774459134s) [upload=497533]: Waiting for error group managing segments... Sep 22 03:28:30.430729979 (565h20m5.783515077s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432230315 (565h20m5.785015413s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432851100 (565h20m5.785636198s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432864114 (565h20m5.785649212s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433283611 (565h20m5.78606871s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433312286 (565h20m5.786097384s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433608802 (565h20m5.7863939s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433900248 (565h20m5.786685346s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434889345 (565h20m5.787674443s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434926946 (565h20m5.787712043s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.435204566 (565h20m5.787989665s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.436237565 (565h20m5.789022653s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:29:00.291216686 (565h20m35.644001784s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291236933 (565h20m35.644022042s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291272671 (565h20m35.644057769s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291291958 (565h20m35.644077066s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291307757 (565h20m35.644092865s) [seg_pos={0 0}, upload=497533]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:29:00.309425256 (565h20m35.662210354s) [upload=497533]: Done uploading segments... Sep 22 03:29:00.309440985 (565h20m35.662226093s) [upload=497533]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 877.475536ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X...X.......XX..........XX.....X..X...X..........X...X.XX...] duration: 417.195822ms FAIL (2025-09-22 03:30:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 417.21594ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X.......XX...................................] duration: 2.125663924s FAIL (2025-09-22 03:30:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.125680275s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.092443965s FAIL (2025-09-22 03:30:25 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.092465325s core uplink ap1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 779.780502ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:30:20 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 779.81133ms core satellite ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 741.26155ms FAIL (2025-09-22 03:30:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 741.281267ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 744.632068ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:29:58 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 744.657776ms core satellite ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X...X.......XX..........XX.....X..X...X..........X...X.XX..] duration: 401.610713ms FAIL (2025-09-22 03:29:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 401.81709ms edge gateway us-select-1
checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 788.664267ms target: https://ap1.storj.io/login FAIL (2025-09-22 03:29:42 UTC) orbiter warning: Get "https://ap1.storj.io/login": dial tcp 34.2.157.232:443: i/o timeout Duration: 788.68727ms core satellite ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 4.938463378s FAIL (2025-09-22 03:29:30 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.938487964s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X.......XX..................................] duration: 2.234367841s FAIL (2025-09-22 03:29:29 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.234386647s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.XX..X...X........X....................................X..X.] duration: 556.549084ms FAIL (2025-09-22 03:29:24 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:28:30.304830750 (565h20m5.657615848s) [upload=497533]: Starting upload Sep 22 03:28:30.304903216 (565h20m5.657688324s) [upload=497533]: Done starting upload Sep 22 03:28:30.304993495 (565h20m5.657778603s) [upload=497533]: Uploading segments... Sep 22 03:28:30.305151993 (565h20m5.657937101s) [upload=497533]: Got next segment. Inline: false Sep 22 03:28:30.305194232 (565h20m5.65797934s) [upload=497533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:28:30.305198770 (565h20m5.657983878s) [upload=497533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:28:30.328856497 (565h20m5.681641605s) [seg_pos={0 0}, upload=497533]: Begin upload segment... Sep 22 03:28:30.328913333 (565h20m5.681698442s) [seg_pos={0 0}, upload=497533]: Done begin upload segment. Sep 22 03:28:30.328950693 (565h20m5.681735801s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.328961063 (565h20m5.681746161s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329079445 (565h20m5.681864553s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329441154 (565h20m5.682226262s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329457795 (565h20m5.682242903s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329862725 (565h20m5.682647833s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329882192 (565h20m5.68266729s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330250232 (565h20m5.68303534s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330290668 (565h20m5.683075776s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330667806 (565h20m5.683452934s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330673116 (565h20m5.683458224s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331075671 (565h20m5.683860779s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331437830 (565h20m5.684222938s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331789951 (565h20m5.68457507s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.332160136 (565h20m5.684945244s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.337945593 (565h20m5.690730701s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.421652625 (565h20m5.774437724s) [upload=497533]: Next returned nil segment Sep 22 03:28:30.421674036 (565h20m5.774459134s) [upload=497533]: Waiting for error group managing segments... Sep 22 03:28:30.430729979 (565h20m5.783515077s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432230315 (565h20m5.785015413s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432851100 (565h20m5.785636198s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432864114 (565h20m5.785649212s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433283611 (565h20m5.78606871s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433312286 (565h20m5.786097384s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433608802 (565h20m5.7863939s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433900248 (565h20m5.786685346s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434889345 (565h20m5.787674443s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434926946 (565h20m5.787712043s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.435204566 (565h20m5.787989665s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.436237565 (565h20m5.789022653s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:29:00.291216686 (565h20m35.644001784s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291236933 (565h20m35.644022042s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291272671 (565h20m35.644057769s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291291958 (565h20m35.644077066s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291307757 (565h20m35.644092865s) [seg_pos={0 0}, upload=497533]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:29:00.309425256 (565h20m35.662210354s) [upload=497533]: Done uploading segments... Sep 22 03:29:00.309440985 (565h20m35.662226093s) [upload=497533]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 556.574893ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 355.756563ms FAIL (2025-09-22 03:29:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ad=497068]: Starting upload Sep 22 02:57:19.602369921 (564h48m54.955155029s) [upload=497068]: Done starting upload Sep 22 02:57:19.602469568 (564h48m54.955254677s) [upload=497068]: Uploading segments... Sep 22 02:57:19.602726330 (564h48m54.955511428s) [upload=497068]: Got next segment. Inline: false Sep 22 02:57:19.602774871 (564h48m54.955559969s) [upload=497068]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 02:57:19.602780101 (564h48m54.955565219s) [upload=497068]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 02:57:19.645724990 (564h48m54.998510098s) [seg_pos={0 0}, upload=497068]: Begin upload segment... Sep 22 02:57:19.645790614 (564h48m54.998575712s) [seg_pos={0 0}, upload=497068]: Done begin upload segment. Sep 22 02:57:19.645811793 (564h48m54.998596901s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.645831420 (564h48m54.998616518s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646098541 (564h48m54.998883639s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646186856 (564h48m54.998971955s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646321099 (564h48m54.999106198s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646554938 (564h48m54.999340036s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646622525 (564h48m54.999407623s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.646776654 (564h48m54.999561762s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647042442 (564h48m54.999827551s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647366281 (564h48m55.000151389s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647642008 (564h48m55.000427106s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647873091 (564h48m55.000658189s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.647939065 (564h48m55.000724163s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.648291717 (564h48m55.001076816s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652209910 (564h48m55.004995018s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652433619 (564h48m55.005218727s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652681976 (564h48m55.005467073s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.652918220 (564h48m55.005703328s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Uploading piece... Sep 22 02:57:19.722974015 (564h48m55.075759113s) [upload=497068]: Next returned nil segment Sep 22 02:57:19.722987761 (564h48m55.075772859s) [upload=497068]: Waiting for error group managing segments... Sep 22 02:57:19.731288267 (564h48m55.084073375s) [noise=true , address=145.79.227.46:4609 , piece_id=2ZUOLR2THIRT2B5U33MCEJTMHOEYIA63X7PJZWODGRKUZY333POQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.731879286 (564h48m55.084664384s) [noise=true , address=145.79.227.35:4488 , piece_id=FVQ7LCFUW7AMPGQ6BBAJLMFWB3V4PL4NPHU2IBJM7W4S4CDH5A6A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.732663838 (564h48m55.085448946s) [noise=true , address=77.90.31.11:4564 , piece_id=EERPDLL4KY7H5AWILCN7SLOZEUPYMJBDDD7H5Z4R2RWQE7Y3J6MQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733610835 (564h48m55.086395943s) [noise=true , address=145.79.227.25:4636 , piece_id=55F5Q4724LMPWA4GGE5VPYII367G25Q6UBEB42JYHSOITRD34IHA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.733956455 (564h48m55.086741552s) [noise=true , address=145.79.227.47:4277 , piece_id=STF2XIBDA3QQF7VBQO2XZ63XHVMKEZDSIWEGTDJP4LBCW3GYV2CA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734772986 (564h48m55.087558083s) [noise=true , address=145.79.227.33:4902 , piece_id=EJRE4ABO52GW2ZOREEMXL4UKCQ6UB62QH5TMJI5BG7B4NZSF6CIA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.734820946 (564h48m55.087606054s) [noise=true , address=145.79.227.28:4759 , piece_id=ACXUEBF5FM5WTYQOJ5XSBT67WA357TEDL72WTOQZJK3SHA7WJCBQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.736621885 (564h48m55.089406983s) [noise=true , address=145.79.227.49:4364 , piece_id=QJTIZXY4KJQZ72KKZRN2QJNZIVMBIVXV3GNZUNJM6FZUL56ATSFQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.744527511 (564h48m55.097312619s) [noise=true , address=145.79.227.49:4351 , piece_id=FXOFGSDT57XT4C4N5KZMV7SZILQMTL75KZAM5GBYR4VC2OUATNUQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.753478278 (564h48m55.106263377s) [noise=true , address=145.79.227.24:4598 , piece_id=FARPW6UUM54BSRUXVJUE5C5IV4FAEZN2XKZ6YVA5GVKBPH6FOGGA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.757789318 (564h48m55.110574426s) [noise=true , address=145.79.227.34:4917 , piece_id=BJGXDSOVIZVS2TENTCMJESSHRE5M5BS5TARN3WUHI5QMY7CVWD7Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.763604260 (564h48m55.116389358s) [noise=true , address=145.79.227.22:4515 , piece_id=P7OLTUWXJQDDGDOMXQ5BYJSSBTIB2FD5SYVS6UQUNNSYHZ2OAQRA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:57:19.773135106 (564h48m55.125920214s) [noise=true , address=145.79.227.43:4161 , piece_id=YZ3FNK66XJQSMMEGDN636IIMDICOFSJX53OETT3VE2Y2KCIOVG7A, seg_pos={0 0}, upload=497068]: Done uploading piece. err: <nil> Sep 22 02:58:19.587557404 (564h49m54.940342522s) [noise=true , address=145.79.227.13:4813 , piece_id=TGMSYHJOCDKN5JYL6STLSU7TKRU7JTKJWOBPJU5ICSWKXH6ZLOAQ, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587651280 (564h49m54.940436398s) [noise=true , address=145.79.227.3:4183 , piece_id=L25HWLZAULCEFT5YGHPEPZUCKKTQ4GUVKW3CR5OKULC4XOZGVKBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12QbZ2BghpxbZDkYtrcJFMf9FfcEmqXobs2xp9N8252UWHdDAGQ, address:145.79.227.3:4183): protocol: rpc: dial tcp 145.79.227.3:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587681085 (564h49m54.940466214s) [noise=true , address=145.79.227.5:4237 , piece_id=WM64JAKN2AUFVG357SA2TF55QEN6PR27MUFQX7BLUDMBD44EC25Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:1o2jLTwKNSjxjFEmckxkG1c5EuySK3o3sawGEikd2Ttz4CWfLa, address:145.79.227.5:4237): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587750837 (564h49m54.940535964s) [noise=true , address=145.79.227.12:4793 , piece_id=G63ZJRQ4O5X32LFAM7P3T64KKNST77IAKXM3O46QUK42O5NKUQBA, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12CffFfbiSKKBGuksEnNL1NuVRjbkRdVUjoYMJisvUM6pEiaUgJ, address:145.79.227.12:4793): protocol: context deadline exceeded; context deadline exceeded Sep 22 02:58:19.587844532 (564h49m54.94062966s) [noise=true , address=145.79.227.13:4821 , piece_id=6RSGVCHJHEJPCYLFGYFNQHHSNSL7I565DHOPGIYK5GGRQQLFM57Q, seg_pos={0 0}, upload=497068]: Done uploading piece. err: ecclient: upload failed (node:12TwpADh23qQ4v6WajK7wdaDrrTbWrQxRg6oQA48rZBDSxCK9rE, address:145.79.227.13:4821): protocol: rpc: dial tcp 145.79.227.13:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 02:58:19.587902271 (564h49m54.940687439s) [seg_pos={0 0}, upload=497068]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Sep 22 02:58:19.609127882 (564h49m54.961913011s) [upload=497068]: Done uploading segments... Sep 22 02:58:19.609146166 (564h49m54.961931295s) [upload=497068]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 355.775507ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X...X.......XX..........XX.....X..X...X..........X...X.XX.] duration: 466.723431ms FAIL (2025-09-22 03:29:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 466.74939ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.XX..X...X........X....................................X..X] duration: 30.019784752s FAIL (2025-09-22 03:29:00 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:28:30.304830750 (565h20m5.657615848s) [upload=497533]: Starting upload Sep 22 03:28:30.304903216 (565h20m5.657688324s) [upload=497533]: Done starting upload Sep 22 03:28:30.304993495 (565h20m5.657778603s) [upload=497533]: Uploading segments... Sep 22 03:28:30.305151993 (565h20m5.657937101s) [upload=497533]: Got next segment. Inline: false Sep 22 03:28:30.305194232 (565h20m5.65797934s) [upload=497533]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:28:30.305198770 (565h20m5.657983878s) [upload=497533]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:28:30.328856497 (565h20m5.681641605s) [seg_pos={0 0}, upload=497533]: Begin upload segment... Sep 22 03:28:30.328913333 (565h20m5.681698442s) [seg_pos={0 0}, upload=497533]: Done begin upload segment. Sep 22 03:28:30.328950693 (565h20m5.681735801s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.328961063 (565h20m5.681746161s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329079445 (565h20m5.681864553s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329441154 (565h20m5.682226262s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329457795 (565h20m5.682242903s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329862725 (565h20m5.682647833s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.329882192 (565h20m5.68266729s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330250232 (565h20m5.68303534s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330290668 (565h20m5.683075776s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330667806 (565h20m5.683452934s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.330673116 (565h20m5.683458224s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331075671 (565h20m5.683860779s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331437830 (565h20m5.684222938s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.331789951 (565h20m5.68457507s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.332160136 (565h20m5.684945244s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.337945593 (565h20m5.690730701s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Uploading piece... Sep 22 03:28:30.421652625 (565h20m5.774437724s) [upload=497533]: Next returned nil segment Sep 22 03:28:30.421674036 (565h20m5.774459134s) [upload=497533]: Waiting for error group managing segments... Sep 22 03:28:30.430729979 (565h20m5.783515077s) [noise=true , address=45.91.116.11:4119 , piece_id=MWJZTDWPHCHPLXENSD3FOPROSWA7GUQXES42QH4GRP2SSHMHSLQQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432230315 (565h20m5.785015413s) [noise=true , address=77.90.31.11:4555 , piece_id=ZCP3M5NHTV6DYEPGDWSFLJHHFKQQHKGXETHO7VO2BEPSKSRM4JJQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432851100 (565h20m5.785636198s) [noise=true , address=77.90.31.13:4627 , piece_id=QNGDYQN6ZL6X4TXQCS7ARBZEUTIY3CD2TCORLEB7N6YG77ZMLA2A, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.432864114 (565h20m5.785649212s) [noise=true , address=45.91.116.9:4261 , piece_id=7YYWTSZJR36KAO4BBFUVEJ4X2WWWE5IWMS3XNMEGGDGB5NI37RSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433283611 (565h20m5.78606871s) [noise=true , address=77.90.31.12:4592 , piece_id=X7AWXOE4CDGTKZKZWKBMJG36YPZCOOYZ7WGOPZHSFM4A3F75NQNA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433312286 (565h20m5.786097384s) [noise=true , address=77.90.31.19:4842 , piece_id=XAMJF4D3G22LPB4NX4DFDQVRHFV4OY35AXG2LO5B2J5DSNX2SWSQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433608802 (565h20m5.7863939s) [noise=true , address=77.90.31.10:4519 , piece_id=SK6HZCOSY2YCJJAQS27YZMGA5I225Y4PQOYWHW7H5TXYJRUBXWMQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.433900248 (565h20m5.786685346s) [noise=true , address=45.91.116.13:4155 , piece_id=2O7BBJZJCYWMBVOSQLZ3YNPY4SRK6I6O5OCGNFAZAM4M6PYUEWWA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434889345 (565h20m5.787674443s) [noise=true , address=45.91.116.13:4154 , piece_id=SCZMTX742UFBL2LQ5UQJOZDYQTOE7S5HEHJMWGG2XR6KQNVPL5OQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.434926946 (565h20m5.787712043s) [noise=true , address=45.91.116.7:4009 , piece_id=JMLEXAHN4LMM4NNDEEQI6AT3UWD76TP26WQZRXX7WFV47OGZP7AQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.435204566 (565h20m5.787989665s) [noise=true , address=77.90.31.16:4735 , piece_id=3CCVZCQH4DZEKBM4F2VAQMUSVB6AWTSTLCH35JMPUHBA6KROM3HQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:28:30.436237565 (565h20m5.789022653s) [noise=true , address=77.90.31.18:4806 , piece_id=KFRWSKOEQCNFZVPKPR4LQRX25DDP7AMTDJ2AJFEIQTS4OWX6C3TA, seg_pos={0 0}, upload=497533]: Done uploading piece. err: <nil> Sep 22 03:29:00.291216686 (565h20m35.644001784s) [noise=true , address=145.79.227.4:4187 , piece_id=UKYONL2Q7RDAQHTORMDCHEWW7JXP6GUVOQQDAMAKGK3BNHRQKDPQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291236933 (565h20m35.644022042s) [noise=true , address=145.79.227.18:4005 , piece_id=M4A2PPH4VFBE6XCEUANZUDNZAJOJPQE4BYVCAZW3GPSJHXY6JRHQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291272671 (565h20m35.644057769s) [noise=true , address=145.79.227.13:4803 , piece_id=I46SH3NY7OQTYUTXVP7MSDICEFYVUDYTL6KPI2KZOCZ3JINPAXYQ, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:29:00.291291958 (565h20m35.644077066s) [noise=true , address=145.79.227.17:4972 , piece_id=BWSBLXS32AE6D7YJ4BXOLBLS57RXBZ4HDR5NUEQZLAACFTNERW4Q, seg_pos={0 0}, upload=497533]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: rpc: dial tcp 145.79.227.17:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:29:00.291307757 (565h20m35.644092865s) [seg_pos={0 0}, upload=497533]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:29:00.309425256 (565h20m35.662210354s) [upload=497533]: Done uploading segments... Sep 22 03:29:00.309440985 (565h20m35.662226093s) [upload=497533]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.019805351s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.HTTP{"GET" "https://ap1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.ap1.satellite_login)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)