Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X.XX..] duration: 411.607318ms FAIL (2025-08-17 04:24:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 411.628157ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XXXX..] duration: 650.504476ms FAIL (2025-08-17 04:24:47 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== se Aug 17 04:23:28.620840633 (202h57m9.352108284s) [upload=211175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:28.620845953 (202h57m9.352113603s) [upload=211175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:28.646835158 (202h57m9.378102858s) [seg_pos={0 0}, upload=211175]: Begin upload segment... Aug 17 04:23:28.646936819 (202h57m9.378204468s) [seg_pos={0 0}, upload=211175]: Done begin upload segment. Aug 17 04:23:28.646989559 (202h57m9.378257218s) [noise=true , address=104.36.82.49:4003 , piece_id=HTLAF4GFG7OBKTKFYLXF5QMOAPWAEL7ZD2IJ64BO5NRJGEVKIN5Q, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647051715 (202h57m9.378319375s) [noise=true , address=45.91.116.14:4210 , piece_id=B5WJCBD7TMMJV2UZOB2H7PTMOUQU3IKFQULK7RR6GM2YXNHGXTWQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647662535 (202h57m9.378930186s) [noise=true , address=45.91.116.12:4242 , piece_id=2YFHC6NRUPE6S6A6ZEZIFT6WVWPNHNNZJI22DUSX6SIKXOATGHNQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647827836 (202h57m9.379095487s) [noise=true , address=104.36.82.23:4455 , piece_id=BCSYWGESJWL56J63GIUEIV3BRWOO4YRW4UXUMCGB37J53CBMUVSA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648134293 (202h57m9.379401954s) [noise=true , address=104.36.82.3:4790 , piece_id=OKZYIEHAI4GGPWYB5SF3JYN7CFCOU6S66JOLC3ULFJBBYHAPMFCA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648314913 (202h57m9.379582573s) [noise=true , address=77.90.31.13:4647 , piece_id=NFEC7SXBDQORDGZA2AAZIEPWVC4BSS6ZYVEPF4NG2VPHFWOX2KJQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648543414 (202h57m9.379811063s) [noise=true , address=104.36.82.60:4365 , piece_id=X2UBI33FNGXGDXW7JFVUYJ4WMQQVEH2P5W2J2NZ6TNDNP6DJSM2A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648834572 (202h57m9.380102222s) [noise=true , address=104.36.82.9:4025 , piece_id=5UFGQ5VECYEBSBA4CAWQNICGM4CY2WBMIKIWMBE3YIGKZVBO7I6A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648992249 (202h57m9.380259908s) [noise=true , address=45.91.116.7:4035 , piece_id=UJOLVS7QCIOJI7VGOUSZEEF5QFKTAEY5EOOR6LTLUI5XKITDJLIA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649335063 (202h57m9.380602713s) [noise=true , address=77.90.31.18:4810 , piece_id=KKU7XFAW572XIR656DOASOGTTNXXHUCDQM4CP6DLCHMJTGH53SXA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649507127 (202h57m9.380774777s) [noise=true , address=79.127.201.215:4345 , piece_id=NOSCVVA3X5XJ37XUXABNBAVLNCZLS5EAE2RP7SRT2P7L6A2CKBZQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649830226 (202h57m9.381097876s) [noise=true , address=77.90.31.19:4853 , piece_id=BKG3LAQW65IFWIOIINQ7AK3TFISFYVQXWH55LLWSHZGGHYYBSSDA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650003341 (202h57m9.381270992s) [noise=true , address=45.91.116.8:4075 , piece_id=6MXDPEQ5STD2NJFMCZGUE3CJPKG7QV7XLVGIMJTF3AL5TCCUW5QQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650256469 (202h57m9.381524128s) [noise=true , address=45.91.116.6:4976 , piece_id=I7VRMIIQOQH5FTN5KXIDKOHRSLOOWWMIETXVALFZ2TMWYY5RIEFA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650434553 (202h57m9.381702213s) [noise=true , address=79.127.205.246:4288 , piece_id=76LPFGIPIVE3MXAZVAROZSCMEWZWMQDEBKLF77TTJPOOVDLZ7AAQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650709621 (202h57m9.381977282s) [noise=true , address=104.36.82.44:4886 , piece_id=IDBWK46PJZRER44U4RB6LUQAGTZEDMSOPAVRBH2CV56RM3JS3I3A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650857891 (202h57m9.38212554s) [noise=true , address=77.90.31.11:4558 , piece_id=EQVKQ7VF476HIAJ2RS5KOLHKC2G3IGFAOD4EDB3SSVNYXXXXOKSA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.651270246 (202h57m9.382537897s) [noise=true , address=45.91.116.13:4172 , piece_id=NTZMLMVOOUNTLTI2AIJFRVJ5FWESBIXFF7IFZEAESDOY74V3K24A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.768946979 (202h57m9.500214629s) [upload=211175]: Next returned nil segment Aug 17 04:23:28.768974191 (202h57m9.500241841s) [upload=211175]: Waiting for error group managing segments... Aug 17 04:23:28.777572686 (202h57m9.508840347s) [noise=true , address=77.90.31.11:4558 , piece_id=EQVKQ7VF476HIAJ2RS5KOLHKC2G3IGFAOD4EDB3SSVNYXXXXOKSA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.778945461 (202h57m9.510213112s) [noise=true , address=45.91.116.14:4210 , piece_id=B5WJCBD7TMMJV2UZOB2H7PTMOUQU3IKFQULK7RR6GM2YXNHGXTWQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.785896566 (202h57m9.517164206s) [noise=true , address=45.91.116.6:4976 , piece_id=I7VRMIIQOQH5FTN5KXIDKOHRSLOOWWMIETXVALFZ2TMWYY5RIEFA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.793497595 (202h57m9.524765245s) [noise=true , address=45.91.116.8:4075 , piece_id=6MXDPEQ5STD2NJFMCZGUE3CJPKG7QV7XLVGIMJTF3AL5TCCUW5QQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.796220951 (202h57m9.527488592s) [noise=true , address=77.90.31.13:4647 , piece_id=NFEC7SXBDQORDGZA2AAZIEPWVC4BSS6ZYVEPF4NG2VPHFWOX2KJQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.799941495 (202h57m9.531209135s) [noise=true , address=45.91.116.12:4242 , piece_id=2YFHC6NRUPE6S6A6ZEZIFT6WVWPNHNNZJI22DUSX6SIKXOATGHNQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.801171361 (202h57m9.532439001s) [noise=true , address=77.90.31.18:4810 , piece_id=KKU7XFAW572XIR656DOASOGTTNXXHUCDQM4CP6DLCHMJTGH53SXA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.802452863 (202h57m9.533720513s) [noise=true , address=45.91.116.13:4172 , piece_id=NTZMLMVOOUNTLTI2AIJFRVJ5FWESBIXFF7IFZEAESDOY74V3K24A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.822657288 (202h57m9.553924938s) [noise=true , address=45.91.116.7:4035 , piece_id=UJOLVS7QCIOJI7VGOUSZEEF5QFKTAEY5EOOR6LTLUI5XKITDJLIA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.874167434 (202h57m9.605435084s) [noise=true , address=77.90.31.19:4853 , piece_id=BKG3LAQW65IFWIOIINQ7AK3TFISFYVQXWH55LLWSHZGGHYYBSSDA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:29.100609905 (202h57m9.831877555s) [noise=true , address=79.127.201.215:4345 , piece_id=NOSCVVA3X5XJ37XUXABNBAVLNCZLS5EAE2RP7SRT2P7L6A2CKBZQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:29.216981541 (202h57m9.948249171s) [noise=true , address=79.127.205.246:4288 , piece_id=76LPFGIPIVE3MXAZVAROZSCMEWZWMQDEBKLF77TTJPOOVDLZ7AAQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:24:28.606052339 (202h58m9.337319999s) [noise=true , address=104.36.82.44:4886 , piece_id=IDBWK46PJZRER44U4RB6LUQAGTZEDMSOPAVRBH2CV56RM3JS3I3A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR, address:104.36.82.44:4886): protocol: rpc: dial tcp 104.36.82.44:4886: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606141437 (202h58m9.337409087s) [noise=true , address=104.36.82.3:4790 , piece_id=OKZYIEHAI4GGPWYB5SF3JYN7CFCOU6S66JOLC3ULFJBBYHAPMFCA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:104.36.82.3:4790): protocol: context deadline exceeded; context deadline exceeded Aug 17 04:24:28.606261834 (202h58m9.337529484s) [noise=true , address=104.36.82.9:4025 , piece_id=5UFGQ5VECYEBSBA4CAWQNICGM4CY2WBMIKIWMBE3YIGKZVBO7I6A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:58906->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606338818 (202h58m9.337606469s) [noise=true , address=104.36.82.23:4455 , piece_id=BCSYWGESJWL56J63GIUEIV3BRWOO4YRW4UXUMCGB37J53CBMUVSA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm, address:104.36.82.23:4455): protocol: write tcp 5.161.89.252:44290->104.36.82.23:4455: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606369045 (202h58m9.337636696s) [noise=true , address=104.36.82.60:4365 , piece_id=X2UBI33FNGXGDXW7JFVUYJ4WMQQVEH2P5W2J2NZ6TNDNP6DJSM2A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: rpc: dial tcp 104.36.82.60:4365: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606443665 (202h58m9.337711316s) [noise=true , address=104.36.82.49:4003 , piece_id=HTLAF4GFG7OBKTKFYLXF5QMOAPWAEL7ZD2IJ64BO5NRJGEVKIN5Q, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1A8MjPSMJHMxQqBbrjewwyZ4MKTwpvYHh3CE7rP3B9NeYgdfmW, address:104.36.82.49:4003): protocol: context deadline exceeded; context deadline exceeded Aug 17 04:24:28.606480044 (202h58m9.337747704s) [seg_pos={0 0}, upload=211175]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 17 04:24:28.621101719 (202h58m9.352369369s) [upload=211175]: Done uploading segments... Aug 17 04:24:28.621117609 (202h58m9.35238526s) [upload=211175]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 650.534433ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.509664298s FAIL (2025-08-17 04:24:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.509725904s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 32.275895072s FAIL (2025-08-17 04:24:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 04:23:05.076746715 (202h56m45.808014345s) [upload=211167]: Starting upload Aug 17 04:23:05.076819060 (202h56m45.80808669s) [upload=211167]: Done starting upload Aug 17 04:23:05.076858426 (202h56m45.808126045s) [upload=211167]: Uploading segments... Aug 17 04:23:05.077035118 (202h56m45.808302748s) [upload=211167]: Got next segment. Inline: false Aug 17 04:23:05.077065765 (202h56m45.808333395s) [upload=211167]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:05.077069302 (202h56m45.808336922s) [upload=211167]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:05.149595571 (202h56m45.880863191s) [seg_pos={0 0}, upload=211167]: Begin upload segment... Aug 17 04:23:05.149634344 (202h56m45.880901974s) [seg_pos={0 0}, upload=211167]: Done begin upload segment. Aug 17 04:23:05.149659451 (202h56m45.880927081s) [noise=true , address=77.90.31.19:4870 , piece_id=OLLFGZRKVZ6ABXS5Z6LQRI5HG2G356XSXE23LYMXFON5IRKOXIQQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.149699086 (202h56m45.880966716s) [noise=true , address=45.91.116.13:4181 , piece_id=SU23KT5A5GRXOG4Y6PHXGYK2EXLDPYBN5DB5Q27XOO5FJ2YWSQXA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150081695 (202h56m45.881349325s) [noise=true , address=66.23.193.175:4559 , piece_id=QXEZFWPC3G25QKMLGXSPDZH4KIDVTZMK5AF7WY7IR2AOLQ37VFHA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150110339 (202h56m45.88137799s) [noise=true , address=45.91.116.11:4147 , piece_id=O6V3FBZMK7DAIK4SFLZYQ2NVAQYJYG5JXMJJAHS6J2DW6ASNQMCQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150390978 (202h56m45.881658608s) [noise=true , address=45.91.116.7:4031 , piece_id=P7L222FZRXYRIACW4W7CIC5AEWMWRPTO2MIXEUAAZFIS567IJ5EA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150446372 (202h56m45.881714002s) [noise=true , address=79.127.205.246:4283 , piece_id=AQKEHJQK5M6DHLVGT5LKJH6SAAJFYA2YEO7TTI7UBTJ7NHSBTWUA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150693598 (202h56m45.881961228s) [noise=true , address=77.90.31.10:4523 , piece_id=2AV6APC7S3A37H5JTRIZEZAIOZ2TFL2YCLJRUXBHIP2PFEXUJI7Q, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150755163 (202h56m45.882022793s) [noise=true , address=104.36.82.22:4514 , piece_id=73OAAQLS545E64H6VE53NXTAPBX4XJXPZN2EEX53EFUSKIBFTXLQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150983263 (202h56m45.882250893s) [noise=true , address=79.127.201.215:4350 , piece_id=J4FJ64ZNWN6TUH5KUE5Z6FRAE7Q64N4DXXFNXUIL3QNAFDCEBVUA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151079885 (202h56m45.882347515s) [noise=true , address=104.36.82.17:4333 , piece_id=AAKCYXF7BIEVWEYXGFSSRQDCQNXLWYBLQLYNX3364RG4656IED7Q, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151274822 (202h56m45.882542452s) [noise=true , address=104.36.82.38:4696 , piece_id=NMDNAT7RVBIA6YQK24Z7XJQSOTIVOBIY5RIHK4EFWJK6N2ZL4UJQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151365532 (202h56m45.882633163s) [noise=true , address=104.36.82.60:4351 , piece_id=JZSLHYLOWTUYL66X2CP6MCGLDMV3A5TYYMXDDSWXD5PFZ7TIGUPA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151584305 (202h56m45.882851935s) [noise=true , address=79.127.201.214:4309 , piece_id=HE4Y4EIXHL3HASHI5KDXHNT6S3YBNQ3MOCR7TGCM5MWBTAE7H6LA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151603451 (202h56m45.882871081s) [noise=true , address=104.36.82.2:4762 , piece_id=CMIP2M7AMYAMK7MIDZU5XJRKNVG5ELS7DJBHP73FD2HQHOF2XRTQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.152583125 (202h56m45.883850755s) [noise=true , address=104.36.82.23:4446 , piece_id=VSLMTGTUUZSEP6X7SCPALTJFOA7IJJCIKLJHK6TIYR6VJQLHKTOQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.153130696 (202h56m45.884398326s) [noise=true , address=104.36.82.5:4864 , piece_id=QU3BHU64HCXUVPBMCEBIYAWX72Q7SGM2I5YF46GNNDM5LJOD542Q, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.154141749 (202h56m45.88540938s) [noise=true , address=77.90.31.11:4580 , piece_id=H5FGTVQYA7SJBS7WCGDLX5W5I6BQBXNQG3GKJLUM6PHXKAE6G5YQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.154561108 (202h56m45.885828738s) [noise=true , address=104.36.82.23:4461 , piece_id=JWAHLP4YYBWYQUWEQC5EHJZZZPVYOUTGC2TS5LNXP4SG7EKJFYYQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.222341082 (202h56m45.953608762s) [upload=211167]: Next returned nil segment Aug 17 04:23:05.222354026 (202h56m45.953621646s) [upload=211167]: Waiting for error group managing segments... Aug 17 04:23:05.230308020 (202h56m45.96157565s) [noise=true , address=45.91.116.7:4031 , piece_id=P7L222FZRXYRIACW4W7CIC5AEWMWRPTO2MIXEUAAZFIS567IJ5EA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.230754350 (202h56m45.96202198s) [noise=true , address=45.91.116.11:4147 , piece_id=O6V3FBZMK7DAIK4SFLZYQ2NVAQYJYG5JXMJJAHS6J2DW6ASNQMCQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.231421757 (202h56m45.962689387s) [noise=true , address=77.90.31.11:4580 , piece_id=H5FGTVQYA7SJBS7WCGDLX5W5I6BQBXNQG3GKJLUM6PHXKAE6G5YQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.231643444 (202h56m45.962911074s) [noise=true , address=45.91.116.13:4181 , piece_id=SU23KT5A5GRXOG4Y6PHXGYK2EXLDPYBN5DB5Q27XOO5FJ2YWSQXA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.296297613 (202h56m46.027565233s) [noise=true , address=77.90.31.10:4523 , piece_id=2AV6APC7S3A37H5JTRIZEZAIOZ2TFL2YCLJRUXBHIP2PFEXUJI7Q, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.346576712 (202h56m46.077844332s) [noise=true , address=77.90.31.19:4870 , piece_id=OLLFGZRKVZ6ABXS5Z6LQRI5HG2G356XSXE23LYMXFON5IRKOXIQQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.494438107 (202h56m46.225705727s) [noise=true , address=79.127.201.214:4309 , piece_id=HE4Y4EIXHL3HASHI5KDXHNT6S3YBNQ3MOCR7TGCM5MWBTAE7H6LA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.563775022 (202h56m46.295042642s) [noise=true , address=79.127.201.215:4350 , piece_id=J4FJ64ZNWN6TUH5KUE5Z6FRAE7Q64N4DXXFNXUIL3QNAFDCEBVUA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.674589117 (202h56m46.405856747s) [noise=true , address=79.127.205.246:4283 , piece_id=AQKEHJQK5M6DHLVGT5LKJH6SAAJFYA2YEO7TTI7UBTJ7NHSBTWUA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.747386525 (202h56m46.478654145s) [noise=true , address=66.23.193.175:4559 , piece_id=QXEZFWPC3G25QKMLGXSPDZH4KIDVTZMK5AF7WY7IR2AOLQ37VFHA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:24:05.061032967 (202h57m45.792300617s) [noise=true , address=104.36.82.60:4351 , piece_id=JZSLHYLOWTUYL66X2CP6MCGLDMV3A5TYYMXDDSWXD5PFZ7TIGUPA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg, address:104.36.82.60:4351): protocol: rpc: dial tcp 104.36.82.60:4351: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061097619 (202h57m45.79236527s) [noise=true , address=104.36.82.38:4696 , piece_id=NMDNAT7RVBIA6YQK24Z7XJQSOTIVOBIY5RIHK4EFWJK6N2ZL4UJQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: rpc: dial tcp 104.36.82.38:4696: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061148564 (202h57m45.792416215s) [noise=true , address=104.36.82.17:4333 , piece_id=AAKCYXF7BIEVWEYXGFSSRQDCQNXLWYBLQLYNX3364RG4656IED7Q, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: rpc: dial tcp 104.36.82.17:4333: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061255446 (202h57m45.792523106s) [noise=true , address=104.36.82.22:4514 , piece_id=73OAAQLS545E64H6VE53NXTAPBX4XJXPZN2EEX53EFUSKIBFTXLQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:104.36.82.22:4514): protocol: rpc: dial tcp 104.36.82.22:4514: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061322663 (202h57m45.792590312s) [noise=true , address=104.36.82.23:4446 , piece_id=VSLMTGTUUZSEP6X7SCPALTJFOA7IJJCIKLJHK6TIYR6VJQLHKTOQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:104.36.82.23:4446): protocol: write tcp 5.161.89.252:28480->104.36.82.23:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061363470 (202h57m45.79263112s) [noise=true , address=104.36.82.2:4762 , piece_id=CMIP2M7AMYAMK7MIDZU5XJRKNVG5ELS7DJBHP73FD2HQHOF2XRTQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: rpc: dial tcp 104.36.82.2:4762: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061387064 (202h57m45.792654714s) [noise=true , address=104.36.82.5:4864 , piece_id=QU3BHU64HCXUVPBMCEBIYAWX72Q7SGM2I5YF46GNNDM5LJOD542Q, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u, address:104.36.82.5:4864): protocol: rpc: dial tcp 104.36.82.5:4864: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061424123 (202h57m45.792691784s) [noise=true , address=104.36.82.23:4461 , piece_id=JWAHLP4YYBWYQUWEQC5EHJZZZPVYOUTGC2TS5LNXP4SG7EKJFYYQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:104.36.82.23:4461): protocol: write tcp 5.161.89.252:18768->104.36.82.23:4461: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061453258 (202h57m45.792720908s) [seg_pos={0 0}, upload=211167]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:24:05.085107047 (202h57m45.816374697s) [upload=211167]: Done uploading segments... Aug 17 04:24:05.085125321 (202h57m45.816392971s) [upload=211167]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 32.275917635s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXXXXX.] duration: 11.722841088s FAIL (2025-08-17 04:24:32 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:07.365016247 (202h56m48.096283887s) [upload=211168]: Starting upload Aug 17 04:23:07.365113880 (202h56m48.09638152s) [upload=211168]: Done starting upload Aug 17 04:23:07.365185295 (202h56m48.096452915s) [upload=211168]: Uploading segments... Aug 17 04:23:07.365444052 (202h56m48.096711693s) [upload=211168]: Got next segment. Inline: false Aug 17 04:23:07.365496200 (202h56m48.09676384s) [upload=211168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:07.365506590 (202h56m48.096774229s) [upload=211168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:07.391532593 (202h56m48.122800213s) [seg_pos={0 0}, upload=211168]: Begin upload segment... Aug 17 04:23:07.391561247 (202h56m48.122828867s) [seg_pos={0 0}, upload=211168]: Done begin upload segment. Aug 17 04:23:07.391580673 (202h56m48.122848294s) [noise=true , address=45.91.116.4:4938 , piece_id=F3AVEZ2YIU4EJZCAGSWZGJPNJ7KNSRGRHMNMLPE7XQLMXWGL62XQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.391635377 (202h56m48.122903017s) [noise=true , address=45.91.116.5:4298 , piece_id=SO6VVAFXQBPJMLOQOEFGGC4D6ICGBUK7BVMKHS4D3HYHCQWABJXA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.391895125 (202h56m48.123162745s) [noise=true , address=45.91.116.4:4937 , piece_id=BEYPSNQUECATWBTX4JUM6NP2BBFXQSYVFBNMJFO5O4UNHRLTZLJA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392097867 (202h56m48.123365508s) [noise=true , address=45.91.116.11:4117 , piece_id=OY5KNPDA5EI6OIMR5DBQ7IHSEJQK777NPIKWOZI3RZH7NIF7GWAQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392117784 (202h56m48.123385405s) [noise=true , address=104.36.82.16:4295 , piece_id=7BPNXEZ5JSW4T6DUZED3BAGJZLUAGN2PGDHSXLUV34AC7BM3OGYA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392313993 (202h56m48.123581614s) [noise=true , address=45.91.116.7:4009 , piece_id=36E4LB3QDRVIJ7V3CFTBEU7XVJNMHPXMO3W4U2EWNRRKPQY6RNBQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392395668 (202h56m48.123663308s) [noise=true , address=45.91.116.13:4154 , piece_id=NGDMGFLNRFPCKKQR72KBGJJXOTE5CV2W6OEQAWRJKCTENPUH3T5A, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392493893 (202h56m48.123761513s) [noise=true , address=104.36.82.4:4803 , piece_id=TV3P4HUUHIXIJ7P2ZONBQPAJAY3UZVVLFF4P7TXAZEU47Y6RJ4QQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392681466 (202h56m48.123949106s) [noise=true , address=77.90.31.19:4841 , piece_id=L7TJCCUKL4RISGHZFGOFLKVNKJ3LK2PAAFWQCRJAV7N5VTNPSWDQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392702645 (202h56m48.123970275s) [noise=true , address=104.36.82.12:4151 , piece_id=3Q6OWEBKFSXBCK5AJ6NHFUIO2GXBO3A6QX7X5T2VNQK42KI5KLNQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392977794 (202h56m48.124245414s) [noise=true , address=104.36.82.2:4730 , piece_id=NPIC6AKOB2DBFDFK36KDDJANJZN7JFTISF6DI2K4OTCJIOABPK6Q, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392988844 (202h56m48.124256475s) [noise=true , address=104.36.82.5:4839 , piece_id=5C6WOWRH4JAPDZFHZYVT7L5GMKU2MYQGWQLIVRUUJJHOOD73HFFA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393185134 (202h56m48.124452754s) [noise=true , address=45.91.116.14:4189 , piece_id=JRIG76HQYCBKFQ67GEXATVH3T5C5I6VLJWNU6XNSKRR5YIAF6UJA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393273310 (202h56m48.12454094s) [noise=true , address=104.36.82.8:4969 , piece_id=ZP65DV3U7BGQ4OHGXJWB6NSWJBIZIEFLERF7BNBIMTGH5YQWBTNQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393400339 (202h56m48.124667959s) [noise=true , address=45.91.116.10:4081 , piece_id=F4SEIIXTTLFASZHDJBYUBDYLJMAZC6K7MEAIPD3WAQ35UHAEAZMQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393572042 (202h56m48.124839683s) [noise=true , address=45.91.116.7:4010 , piece_id=6RWG7YB22HSREMJA44RC43SBH3TPQ3F3EDKAC2JULNK2SWSBWNEQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.461635871 (202h56m48.192903501s) [upload=211168]: Next returned nil segment Aug 17 04:23:07.461646972 (202h56m48.192914602s) [upload=211168]: Waiting for error group managing segments... Aug 17 04:23:07.469880881 (202h56m48.201148501s) [noise=true , address=45.91.116.4:4938 , piece_id=F3AVEZ2YIU4EJZCAGSWZGJPNJ7KNSRGRHMNMLPE7XQLMXWGL62XQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470249144 (202h56m48.201516775s) [noise=true , address=45.91.116.13:4154 , piece_id=NGDMGFLNRFPCKKQR72KBGJJXOTE5CV2W6OEQAWRJKCTENPUH3T5A, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470491772 (202h56m48.201759402s) [noise=true , address=45.91.116.7:4009 , piece_id=36E4LB3QDRVIJ7V3CFTBEU7XVJNMHPXMO3W4U2EWNRRKPQY6RNBQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470580539 (202h56m48.201848169s) [noise=true , address=45.91.116.4:4937 , piece_id=BEYPSNQUECATWBTX4JUM6NP2BBFXQSYVFBNMJFO5O4UNHRLTZLJA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470934566 (202h56m48.202202196s) [noise=true , address=45.91.116.14:4189 , piece_id=JRIG76HQYCBKFQ67GEXATVH3T5C5I6VLJWNU6XNSKRR5YIAF6UJA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471147235 (202h56m48.202414855s) [noise=true , address=45.91.116.10:4081 , piece_id=F4SEIIXTTLFASZHDJBYUBDYLJMAZC6K7MEAIPD3WAQ35UHAEAZMQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471257373 (202h56m48.202525003s) [noise=true , address=45.91.116.5:4298 , piece_id=SO6VVAFXQBPJMLOQOEFGGC4D6ICGBUK7BVMKHS4D3HYHCQWABJXA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471574630 (202h56m48.20284225s) [noise=true , address=45.91.116.7:4010 , piece_id=6RWG7YB22HSREMJA44RC43SBH3TPQ3F3EDKAC2JULNK2SWSBWNEQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471656044 (202h56m48.202923674s) [noise=true , address=45.91.116.11:4117 , piece_id=OY5KNPDA5EI6OIMR5DBQ7IHSEJQK777NPIKWOZI3RZH7NIF7GWAQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.472781833 (202h56m48.204049453s) [noise=true , address=77.90.31.19:4841 , piece_id=L7TJCCUKL4RISGHZFGOFLKVNKJ3LK2PAAFWQCRJAV7N5VTNPSWDQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:37.353223355 (202h57m18.084491014s) [noise=true , address=104.36.82.4:4803 , piece_id=TV3P4HUUHIXIJ7P2ZONBQPAJAY3UZVVLFF4P7TXAZEU47Y6RJ4QQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: rpc: dial tcp 104.36.82.4:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353345955 (202h57m18.084613605s) [noise=true , address=104.36.82.5:4839 , piece_id=5C6WOWRH4JAPDZFHZYVT7L5GMKU2MYQGWQLIVRUUJJHOOD73HFFA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: rpc: dial tcp 104.36.82.5:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353365382 (202h57m18.084633042s) [noise=true , address=104.36.82.12:4151 , piece_id=3Q6OWEBKFSXBCK5AJ6NHFUIO2GXBO3A6QX7X5T2VNQK42KI5KLNQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353416519 (202h57m18.084684169s) [noise=true , address=104.36.82.2:4730 , piece_id=NPIC6AKOB2DBFDFK36KDDJANJZN7JFTISF6DI2K4OTCJIOABPK6Q, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353477142 (202h57m18.084744793s) [noise=true , address=104.36.82.16:4295 , piece_id=7BPNXEZ5JSW4T6DUZED3BAGJZLUAGN2PGDHSXLUV34AC7BM3OGYA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 5.161.89.252:26914->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353487252 (202h57m18.084754901s) [noise=true , address=104.36.82.8:4969 , piece_id=ZP65DV3U7BGQ4OHGXJWB6NSWJBIZIEFLERF7BNBIMTGH5YQWBTNQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: rpc: dial tcp 104.36.82.8:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353530393 (202h57m18.084798093s) [seg_pos={0 0}, upload=211168]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:37.392615539 (202h57m18.123883199s) [upload=211168]: Done uploading segments... Aug 17 04:23:37.392649854 (202h57m18.123917504s) [upload=211168]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 11.722865003s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 1.428331864s FAIL (2025-08-17 04:24:30 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== se Aug 17 04:23:28.620840633 (202h57m9.352108284s) [upload=211175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:28.620845953 (202h57m9.352113603s) [upload=211175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:28.646835158 (202h57m9.378102858s) [seg_pos={0 0}, upload=211175]: Begin upload segment... Aug 17 04:23:28.646936819 (202h57m9.378204468s) [seg_pos={0 0}, upload=211175]: Done begin upload segment. Aug 17 04:23:28.646989559 (202h57m9.378257218s) [noise=true , address=104.36.82.49:4003 , piece_id=HTLAF4GFG7OBKTKFYLXF5QMOAPWAEL7ZD2IJ64BO5NRJGEVKIN5Q, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647051715 (202h57m9.378319375s) [noise=true , address=45.91.116.14:4210 , piece_id=B5WJCBD7TMMJV2UZOB2H7PTMOUQU3IKFQULK7RR6GM2YXNHGXTWQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647662535 (202h57m9.378930186s) [noise=true , address=45.91.116.12:4242 , piece_id=2YFHC6NRUPE6S6A6ZEZIFT6WVWPNHNNZJI22DUSX6SIKXOATGHNQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647827836 (202h57m9.379095487s) [noise=true , address=104.36.82.23:4455 , piece_id=BCSYWGESJWL56J63GIUEIV3BRWOO4YRW4UXUMCGB37J53CBMUVSA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648134293 (202h57m9.379401954s) [noise=true , address=104.36.82.3:4790 , piece_id=OKZYIEHAI4GGPWYB5SF3JYN7CFCOU6S66JOLC3ULFJBBYHAPMFCA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648314913 (202h57m9.379582573s) [noise=true , address=77.90.31.13:4647 , piece_id=NFEC7SXBDQORDGZA2AAZIEPWVC4BSS6ZYVEPF4NG2VPHFWOX2KJQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648543414 (202h57m9.379811063s) [noise=true , address=104.36.82.60:4365 , piece_id=X2UBI33FNGXGDXW7JFVUYJ4WMQQVEH2P5W2J2NZ6TNDNP6DJSM2A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648834572 (202h57m9.380102222s) [noise=true , address=104.36.82.9:4025 , piece_id=5UFGQ5VECYEBSBA4CAWQNICGM4CY2WBMIKIWMBE3YIGKZVBO7I6A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648992249 (202h57m9.380259908s) [noise=true , address=45.91.116.7:4035 , piece_id=UJOLVS7QCIOJI7VGOUSZEEF5QFKTAEY5EOOR6LTLUI5XKITDJLIA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649335063 (202h57m9.380602713s) [noise=true , address=77.90.31.18:4810 , piece_id=KKU7XFAW572XIR656DOASOGTTNXXHUCDQM4CP6DLCHMJTGH53SXA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649507127 (202h57m9.380774777s) [noise=true , address=79.127.201.215:4345 , piece_id=NOSCVVA3X5XJ37XUXABNBAVLNCZLS5EAE2RP7SRT2P7L6A2CKBZQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649830226 (202h57m9.381097876s) [noise=true , address=77.90.31.19:4853 , piece_id=BKG3LAQW65IFWIOIINQ7AK3TFISFYVQXWH55LLWSHZGGHYYBSSDA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650003341 (202h57m9.381270992s) [noise=true , address=45.91.116.8:4075 , piece_id=6MXDPEQ5STD2NJFMCZGUE3CJPKG7QV7XLVGIMJTF3AL5TCCUW5QQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650256469 (202h57m9.381524128s) [noise=true , address=45.91.116.6:4976 , piece_id=I7VRMIIQOQH5FTN5KXIDKOHRSLOOWWMIETXVALFZ2TMWYY5RIEFA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650434553 (202h57m9.381702213s) [noise=true , address=79.127.205.246:4288 , piece_id=76LPFGIPIVE3MXAZVAROZSCMEWZWMQDEBKLF77TTJPOOVDLZ7AAQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650709621 (202h57m9.381977282s) [noise=true , address=104.36.82.44:4886 , piece_id=IDBWK46PJZRER44U4RB6LUQAGTZEDMSOPAVRBH2CV56RM3JS3I3A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650857891 (202h57m9.38212554s) [noise=true , address=77.90.31.11:4558 , piece_id=EQVKQ7VF476HIAJ2RS5KOLHKC2G3IGFAOD4EDB3SSVNYXXXXOKSA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.651270246 (202h57m9.382537897s) [noise=true , address=45.91.116.13:4172 , piece_id=NTZMLMVOOUNTLTI2AIJFRVJ5FWESBIXFF7IFZEAESDOY74V3K24A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.768946979 (202h57m9.500214629s) [upload=211175]: Next returned nil segment Aug 17 04:23:28.768974191 (202h57m9.500241841s) [upload=211175]: Waiting for error group managing segments... Aug 17 04:23:28.777572686 (202h57m9.508840347s) [noise=true , address=77.90.31.11:4558 , piece_id=EQVKQ7VF476HIAJ2RS5KOLHKC2G3IGFAOD4EDB3SSVNYXXXXOKSA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.778945461 (202h57m9.510213112s) [noise=true , address=45.91.116.14:4210 , piece_id=B5WJCBD7TMMJV2UZOB2H7PTMOUQU3IKFQULK7RR6GM2YXNHGXTWQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.785896566 (202h57m9.517164206s) [noise=true , address=45.91.116.6:4976 , piece_id=I7VRMIIQOQH5FTN5KXIDKOHRSLOOWWMIETXVALFZ2TMWYY5RIEFA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.793497595 (202h57m9.524765245s) [noise=true , address=45.91.116.8:4075 , piece_id=6MXDPEQ5STD2NJFMCZGUE3CJPKG7QV7XLVGIMJTF3AL5TCCUW5QQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.796220951 (202h57m9.527488592s) [noise=true , address=77.90.31.13:4647 , piece_id=NFEC7SXBDQORDGZA2AAZIEPWVC4BSS6ZYVEPF4NG2VPHFWOX2KJQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.799941495 (202h57m9.531209135s) [noise=true , address=45.91.116.12:4242 , piece_id=2YFHC6NRUPE6S6A6ZEZIFT6WVWPNHNNZJI22DUSX6SIKXOATGHNQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.801171361 (202h57m9.532439001s) [noise=true , address=77.90.31.18:4810 , piece_id=KKU7XFAW572XIR656DOASOGTTNXXHUCDQM4CP6DLCHMJTGH53SXA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.802452863 (202h57m9.533720513s) [noise=true , address=45.91.116.13:4172 , piece_id=NTZMLMVOOUNTLTI2AIJFRVJ5FWESBIXFF7IFZEAESDOY74V3K24A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.822657288 (202h57m9.553924938s) [noise=true , address=45.91.116.7:4035 , piece_id=UJOLVS7QCIOJI7VGOUSZEEF5QFKTAEY5EOOR6LTLUI5XKITDJLIA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.874167434 (202h57m9.605435084s) [noise=true , address=77.90.31.19:4853 , piece_id=BKG3LAQW65IFWIOIINQ7AK3TFISFYVQXWH55LLWSHZGGHYYBSSDA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:29.100609905 (202h57m9.831877555s) [noise=true , address=79.127.201.215:4345 , piece_id=NOSCVVA3X5XJ37XUXABNBAVLNCZLS5EAE2RP7SRT2P7L6A2CKBZQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:29.216981541 (202h57m9.948249171s) [noise=true , address=79.127.205.246:4288 , piece_id=76LPFGIPIVE3MXAZVAROZSCMEWZWMQDEBKLF77TTJPOOVDLZ7AAQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:24:28.606052339 (202h58m9.337319999s) [noise=true , address=104.36.82.44:4886 , piece_id=IDBWK46PJZRER44U4RB6LUQAGTZEDMSOPAVRBH2CV56RM3JS3I3A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR, address:104.36.82.44:4886): protocol: rpc: dial tcp 104.36.82.44:4886: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606141437 (202h58m9.337409087s) [noise=true , address=104.36.82.3:4790 , piece_id=OKZYIEHAI4GGPWYB5SF3JYN7CFCOU6S66JOLC3ULFJBBYHAPMFCA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:104.36.82.3:4790): protocol: context deadline exceeded; context deadline exceeded Aug 17 04:24:28.606261834 (202h58m9.337529484s) [noise=true , address=104.36.82.9:4025 , piece_id=5UFGQ5VECYEBSBA4CAWQNICGM4CY2WBMIKIWMBE3YIGKZVBO7I6A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:58906->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606338818 (202h58m9.337606469s) [noise=true , address=104.36.82.23:4455 , piece_id=BCSYWGESJWL56J63GIUEIV3BRWOO4YRW4UXUMCGB37J53CBMUVSA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm, address:104.36.82.23:4455): protocol: write tcp 5.161.89.252:44290->104.36.82.23:4455: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606369045 (202h58m9.337636696s) [noise=true , address=104.36.82.60:4365 , piece_id=X2UBI33FNGXGDXW7JFVUYJ4WMQQVEH2P5W2J2NZ6TNDNP6DJSM2A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: rpc: dial tcp 104.36.82.60:4365: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606443665 (202h58m9.337711316s) [noise=true , address=104.36.82.49:4003 , piece_id=HTLAF4GFG7OBKTKFYLXF5QMOAPWAEL7ZD2IJ64BO5NRJGEVKIN5Q, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1A8MjPSMJHMxQqBbrjewwyZ4MKTwpvYHh3CE7rP3B9NeYgdfmW, address:104.36.82.49:4003): protocol: context deadline exceeded; context deadline exceeded Aug 17 04:24:28.606480044 (202h58m9.337747704s) [seg_pos={0 0}, upload=211175]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 17 04:24:28.621101719 (202h58m9.352369369s) [upload=211175]: Done uploading segments... Aug 17 04:24:28.621117609 (202h58m9.35238526s) [upload=211175]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1.428349557s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.017321231s FAIL (2025-08-17 04:24:28 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== se Aug 17 04:23:28.620840633 (202h57m9.352108284s) [upload=211175]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:28.620845953 (202h57m9.352113603s) [upload=211175]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:28.646835158 (202h57m9.378102858s) [seg_pos={0 0}, upload=211175]: Begin upload segment... Aug 17 04:23:28.646936819 (202h57m9.378204468s) [seg_pos={0 0}, upload=211175]: Done begin upload segment. Aug 17 04:23:28.646989559 (202h57m9.378257218s) [noise=true , address=104.36.82.49:4003 , piece_id=HTLAF4GFG7OBKTKFYLXF5QMOAPWAEL7ZD2IJ64BO5NRJGEVKIN5Q, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647051715 (202h57m9.378319375s) [noise=true , address=45.91.116.14:4210 , piece_id=B5WJCBD7TMMJV2UZOB2H7PTMOUQU3IKFQULK7RR6GM2YXNHGXTWQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647662535 (202h57m9.378930186s) [noise=true , address=45.91.116.12:4242 , piece_id=2YFHC6NRUPE6S6A6ZEZIFT6WVWPNHNNZJI22DUSX6SIKXOATGHNQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.647827836 (202h57m9.379095487s) [noise=true , address=104.36.82.23:4455 , piece_id=BCSYWGESJWL56J63GIUEIV3BRWOO4YRW4UXUMCGB37J53CBMUVSA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648134293 (202h57m9.379401954s) [noise=true , address=104.36.82.3:4790 , piece_id=OKZYIEHAI4GGPWYB5SF3JYN7CFCOU6S66JOLC3ULFJBBYHAPMFCA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648314913 (202h57m9.379582573s) [noise=true , address=77.90.31.13:4647 , piece_id=NFEC7SXBDQORDGZA2AAZIEPWVC4BSS6ZYVEPF4NG2VPHFWOX2KJQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648543414 (202h57m9.379811063s) [noise=true , address=104.36.82.60:4365 , piece_id=X2UBI33FNGXGDXW7JFVUYJ4WMQQVEH2P5W2J2NZ6TNDNP6DJSM2A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648834572 (202h57m9.380102222s) [noise=true , address=104.36.82.9:4025 , piece_id=5UFGQ5VECYEBSBA4CAWQNICGM4CY2WBMIKIWMBE3YIGKZVBO7I6A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.648992249 (202h57m9.380259908s) [noise=true , address=45.91.116.7:4035 , piece_id=UJOLVS7QCIOJI7VGOUSZEEF5QFKTAEY5EOOR6LTLUI5XKITDJLIA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649335063 (202h57m9.380602713s) [noise=true , address=77.90.31.18:4810 , piece_id=KKU7XFAW572XIR656DOASOGTTNXXHUCDQM4CP6DLCHMJTGH53SXA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649507127 (202h57m9.380774777s) [noise=true , address=79.127.201.215:4345 , piece_id=NOSCVVA3X5XJ37XUXABNBAVLNCZLS5EAE2RP7SRT2P7L6A2CKBZQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.649830226 (202h57m9.381097876s) [noise=true , address=77.90.31.19:4853 , piece_id=BKG3LAQW65IFWIOIINQ7AK3TFISFYVQXWH55LLWSHZGGHYYBSSDA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650003341 (202h57m9.381270992s) [noise=true , address=45.91.116.8:4075 , piece_id=6MXDPEQ5STD2NJFMCZGUE3CJPKG7QV7XLVGIMJTF3AL5TCCUW5QQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650256469 (202h57m9.381524128s) [noise=true , address=45.91.116.6:4976 , piece_id=I7VRMIIQOQH5FTN5KXIDKOHRSLOOWWMIETXVALFZ2TMWYY5RIEFA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650434553 (202h57m9.381702213s) [noise=true , address=79.127.205.246:4288 , piece_id=76LPFGIPIVE3MXAZVAROZSCMEWZWMQDEBKLF77TTJPOOVDLZ7AAQ, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650709621 (202h57m9.381977282s) [noise=true , address=104.36.82.44:4886 , piece_id=IDBWK46PJZRER44U4RB6LUQAGTZEDMSOPAVRBH2CV56RM3JS3I3A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.650857891 (202h57m9.38212554s) [noise=true , address=77.90.31.11:4558 , piece_id=EQVKQ7VF476HIAJ2RS5KOLHKC2G3IGFAOD4EDB3SSVNYXXXXOKSA, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.651270246 (202h57m9.382537897s) [noise=true , address=45.91.116.13:4172 , piece_id=NTZMLMVOOUNTLTI2AIJFRVJ5FWESBIXFF7IFZEAESDOY74V3K24A, seg_pos={0 0}, upload=211175]: Uploading piece... Aug 17 04:23:28.768946979 (202h57m9.500214629s) [upload=211175]: Next returned nil segment Aug 17 04:23:28.768974191 (202h57m9.500241841s) [upload=211175]: Waiting for error group managing segments... Aug 17 04:23:28.777572686 (202h57m9.508840347s) [noise=true , address=77.90.31.11:4558 , piece_id=EQVKQ7VF476HIAJ2RS5KOLHKC2G3IGFAOD4EDB3SSVNYXXXXOKSA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.778945461 (202h57m9.510213112s) [noise=true , address=45.91.116.14:4210 , piece_id=B5WJCBD7TMMJV2UZOB2H7PTMOUQU3IKFQULK7RR6GM2YXNHGXTWQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.785896566 (202h57m9.517164206s) [noise=true , address=45.91.116.6:4976 , piece_id=I7VRMIIQOQH5FTN5KXIDKOHRSLOOWWMIETXVALFZ2TMWYY5RIEFA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.793497595 (202h57m9.524765245s) [noise=true , address=45.91.116.8:4075 , piece_id=6MXDPEQ5STD2NJFMCZGUE3CJPKG7QV7XLVGIMJTF3AL5TCCUW5QQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.796220951 (202h57m9.527488592s) [noise=true , address=77.90.31.13:4647 , piece_id=NFEC7SXBDQORDGZA2AAZIEPWVC4BSS6ZYVEPF4NG2VPHFWOX2KJQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.799941495 (202h57m9.531209135s) [noise=true , address=45.91.116.12:4242 , piece_id=2YFHC6NRUPE6S6A6ZEZIFT6WVWPNHNNZJI22DUSX6SIKXOATGHNQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.801171361 (202h57m9.532439001s) [noise=true , address=77.90.31.18:4810 , piece_id=KKU7XFAW572XIR656DOASOGTTNXXHUCDQM4CP6DLCHMJTGH53SXA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.802452863 (202h57m9.533720513s) [noise=true , address=45.91.116.13:4172 , piece_id=NTZMLMVOOUNTLTI2AIJFRVJ5FWESBIXFF7IFZEAESDOY74V3K24A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.822657288 (202h57m9.553924938s) [noise=true , address=45.91.116.7:4035 , piece_id=UJOLVS7QCIOJI7VGOUSZEEF5QFKTAEY5EOOR6LTLUI5XKITDJLIA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:28.874167434 (202h57m9.605435084s) [noise=true , address=77.90.31.19:4853 , piece_id=BKG3LAQW65IFWIOIINQ7AK3TFISFYVQXWH55LLWSHZGGHYYBSSDA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:29.100609905 (202h57m9.831877555s) [noise=true , address=79.127.201.215:4345 , piece_id=NOSCVVA3X5XJ37XUXABNBAVLNCZLS5EAE2RP7SRT2P7L6A2CKBZQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:23:29.216981541 (202h57m9.948249171s) [noise=true , address=79.127.205.246:4288 , piece_id=76LPFGIPIVE3MXAZVAROZSCMEWZWMQDEBKLF77TTJPOOVDLZ7AAQ, seg_pos={0 0}, upload=211175]: Done uploading piece. err: <nil> Aug 17 04:24:28.606052339 (202h58m9.337319999s) [noise=true , address=104.36.82.44:4886 , piece_id=IDBWK46PJZRER44U4RB6LUQAGTZEDMSOPAVRBH2CV56RM3JS3I3A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1n2cXhqd5EWbNPTXem83jQUjYq5ZaGKBidbVCQzxYfR3rWddGR, address:104.36.82.44:4886): protocol: rpc: dial tcp 104.36.82.44:4886: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606141437 (202h58m9.337409087s) [noise=true , address=104.36.82.3:4790 , piece_id=OKZYIEHAI4GGPWYB5SF3JYN7CFCOU6S66JOLC3ULFJBBYHAPMFCA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1AowWwVvxzmnW5qwbqbMBVbvNoigffjAN51LQp8e6XA9CZkeXL, address:104.36.82.3:4790): protocol: context deadline exceeded; context deadline exceeded Aug 17 04:24:28.606261834 (202h58m9.337529484s) [noise=true , address=104.36.82.9:4025 , piece_id=5UFGQ5VECYEBSBA4CAWQNICGM4CY2WBMIKIWMBE3YIGKZVBO7I6A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:58906->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606338818 (202h58m9.337606469s) [noise=true , address=104.36.82.23:4455 , piece_id=BCSYWGESJWL56J63GIUEIV3BRWOO4YRW4UXUMCGB37J53CBMUVSA, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm, address:104.36.82.23:4455): protocol: write tcp 5.161.89.252:44290->104.36.82.23:4455: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606369045 (202h58m9.337636696s) [noise=true , address=104.36.82.60:4365 , piece_id=X2UBI33FNGXGDXW7JFVUYJ4WMQQVEH2P5W2J2NZ6TNDNP6DJSM2A, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a, address:104.36.82.60:4365): protocol: rpc: dial tcp 104.36.82.60:4365: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:28.606443665 (202h58m9.337711316s) [noise=true , address=104.36.82.49:4003 , piece_id=HTLAF4GFG7OBKTKFYLXF5QMOAPWAEL7ZD2IJ64BO5NRJGEVKIN5Q, seg_pos={0 0}, upload=211175]: Done uploading piece. err: ecclient: upload failed (node:1A8MjPSMJHMxQqBbrjewwyZ4MKTwpvYHh3CE7rP3B9NeYgdfmW, address:104.36.82.49:4003): protocol: context deadline exceeded; context deadline exceeded Aug 17 04:24:28.606480044 (202h58m9.337747704s) [seg_pos={0 0}, upload=211175]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 17 04:24:28.621101719 (202h58m9.352369369s) [upload=211175]: Done uploading segments... Aug 17 04:24:28.621117609 (202h58m9.35238526s) [upload=211175]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.017344215s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.026483709s FAIL (2025-08-17 04:24:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 04:23:05.076746715 (202h56m45.808014345s) [upload=211167]: Starting upload Aug 17 04:23:05.076819060 (202h56m45.80808669s) [upload=211167]: Done starting upload Aug 17 04:23:05.076858426 (202h56m45.808126045s) [upload=211167]: Uploading segments... Aug 17 04:23:05.077035118 (202h56m45.808302748s) [upload=211167]: Got next segment. Inline: false Aug 17 04:23:05.077065765 (202h56m45.808333395s) [upload=211167]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:05.077069302 (202h56m45.808336922s) [upload=211167]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:05.149595571 (202h56m45.880863191s) [seg_pos={0 0}, upload=211167]: Begin upload segment... Aug 17 04:23:05.149634344 (202h56m45.880901974s) [seg_pos={0 0}, upload=211167]: Done begin upload segment. Aug 17 04:23:05.149659451 (202h56m45.880927081s) [noise=true , address=77.90.31.19:4870 , piece_id=OLLFGZRKVZ6ABXS5Z6LQRI5HG2G356XSXE23LYMXFON5IRKOXIQQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.149699086 (202h56m45.880966716s) [noise=true , address=45.91.116.13:4181 , piece_id=SU23KT5A5GRXOG4Y6PHXGYK2EXLDPYBN5DB5Q27XOO5FJ2YWSQXA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150081695 (202h56m45.881349325s) [noise=true , address=66.23.193.175:4559 , piece_id=QXEZFWPC3G25QKMLGXSPDZH4KIDVTZMK5AF7WY7IR2AOLQ37VFHA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150110339 (202h56m45.88137799s) [noise=true , address=45.91.116.11:4147 , piece_id=O6V3FBZMK7DAIK4SFLZYQ2NVAQYJYG5JXMJJAHS6J2DW6ASNQMCQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150390978 (202h56m45.881658608s) [noise=true , address=45.91.116.7:4031 , piece_id=P7L222FZRXYRIACW4W7CIC5AEWMWRPTO2MIXEUAAZFIS567IJ5EA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150446372 (202h56m45.881714002s) [noise=true , address=79.127.205.246:4283 , piece_id=AQKEHJQK5M6DHLVGT5LKJH6SAAJFYA2YEO7TTI7UBTJ7NHSBTWUA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150693598 (202h56m45.881961228s) [noise=true , address=77.90.31.10:4523 , piece_id=2AV6APC7S3A37H5JTRIZEZAIOZ2TFL2YCLJRUXBHIP2PFEXUJI7Q, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150755163 (202h56m45.882022793s) [noise=true , address=104.36.82.22:4514 , piece_id=73OAAQLS545E64H6VE53NXTAPBX4XJXPZN2EEX53EFUSKIBFTXLQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.150983263 (202h56m45.882250893s) [noise=true , address=79.127.201.215:4350 , piece_id=J4FJ64ZNWN6TUH5KUE5Z6FRAE7Q64N4DXXFNXUIL3QNAFDCEBVUA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151079885 (202h56m45.882347515s) [noise=true , address=104.36.82.17:4333 , piece_id=AAKCYXF7BIEVWEYXGFSSRQDCQNXLWYBLQLYNX3364RG4656IED7Q, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151274822 (202h56m45.882542452s) [noise=true , address=104.36.82.38:4696 , piece_id=NMDNAT7RVBIA6YQK24Z7XJQSOTIVOBIY5RIHK4EFWJK6N2ZL4UJQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151365532 (202h56m45.882633163s) [noise=true , address=104.36.82.60:4351 , piece_id=JZSLHYLOWTUYL66X2CP6MCGLDMV3A5TYYMXDDSWXD5PFZ7TIGUPA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151584305 (202h56m45.882851935s) [noise=true , address=79.127.201.214:4309 , piece_id=HE4Y4EIXHL3HASHI5KDXHNT6S3YBNQ3MOCR7TGCM5MWBTAE7H6LA, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.151603451 (202h56m45.882871081s) [noise=true , address=104.36.82.2:4762 , piece_id=CMIP2M7AMYAMK7MIDZU5XJRKNVG5ELS7DJBHP73FD2HQHOF2XRTQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.152583125 (202h56m45.883850755s) [noise=true , address=104.36.82.23:4446 , piece_id=VSLMTGTUUZSEP6X7SCPALTJFOA7IJJCIKLJHK6TIYR6VJQLHKTOQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.153130696 (202h56m45.884398326s) [noise=true , address=104.36.82.5:4864 , piece_id=QU3BHU64HCXUVPBMCEBIYAWX72Q7SGM2I5YF46GNNDM5LJOD542Q, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.154141749 (202h56m45.88540938s) [noise=true , address=77.90.31.11:4580 , piece_id=H5FGTVQYA7SJBS7WCGDLX5W5I6BQBXNQG3GKJLUM6PHXKAE6G5YQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.154561108 (202h56m45.885828738s) [noise=true , address=104.36.82.23:4461 , piece_id=JWAHLP4YYBWYQUWEQC5EHJZZZPVYOUTGC2TS5LNXP4SG7EKJFYYQ, seg_pos={0 0}, upload=211167]: Uploading piece... Aug 17 04:23:05.222341082 (202h56m45.953608762s) [upload=211167]: Next returned nil segment Aug 17 04:23:05.222354026 (202h56m45.953621646s) [upload=211167]: Waiting for error group managing segments... Aug 17 04:23:05.230308020 (202h56m45.96157565s) [noise=true , address=45.91.116.7:4031 , piece_id=P7L222FZRXYRIACW4W7CIC5AEWMWRPTO2MIXEUAAZFIS567IJ5EA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.230754350 (202h56m45.96202198s) [noise=true , address=45.91.116.11:4147 , piece_id=O6V3FBZMK7DAIK4SFLZYQ2NVAQYJYG5JXMJJAHS6J2DW6ASNQMCQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.231421757 (202h56m45.962689387s) [noise=true , address=77.90.31.11:4580 , piece_id=H5FGTVQYA7SJBS7WCGDLX5W5I6BQBXNQG3GKJLUM6PHXKAE6G5YQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.231643444 (202h56m45.962911074s) [noise=true , address=45.91.116.13:4181 , piece_id=SU23KT5A5GRXOG4Y6PHXGYK2EXLDPYBN5DB5Q27XOO5FJ2YWSQXA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.296297613 (202h56m46.027565233s) [noise=true , address=77.90.31.10:4523 , piece_id=2AV6APC7S3A37H5JTRIZEZAIOZ2TFL2YCLJRUXBHIP2PFEXUJI7Q, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.346576712 (202h56m46.077844332s) [noise=true , address=77.90.31.19:4870 , piece_id=OLLFGZRKVZ6ABXS5Z6LQRI5HG2G356XSXE23LYMXFON5IRKOXIQQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.494438107 (202h56m46.225705727s) [noise=true , address=79.127.201.214:4309 , piece_id=HE4Y4EIXHL3HASHI5KDXHNT6S3YBNQ3MOCR7TGCM5MWBTAE7H6LA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.563775022 (202h56m46.295042642s) [noise=true , address=79.127.201.215:4350 , piece_id=J4FJ64ZNWN6TUH5KUE5Z6FRAE7Q64N4DXXFNXUIL3QNAFDCEBVUA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.674589117 (202h56m46.405856747s) [noise=true , address=79.127.205.246:4283 , piece_id=AQKEHJQK5M6DHLVGT5LKJH6SAAJFYA2YEO7TTI7UBTJ7NHSBTWUA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:23:05.747386525 (202h56m46.478654145s) [noise=true , address=66.23.193.175:4559 , piece_id=QXEZFWPC3G25QKMLGXSPDZH4KIDVTZMK5AF7WY7IR2AOLQ37VFHA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: <nil> Aug 17 04:24:05.061032967 (202h57m45.792300617s) [noise=true , address=104.36.82.60:4351 , piece_id=JZSLHYLOWTUYL66X2CP6MCGLDMV3A5TYYMXDDSWXD5PFZ7TIGUPA, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg, address:104.36.82.60:4351): protocol: rpc: dial tcp 104.36.82.60:4351: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061097619 (202h57m45.79236527s) [noise=true , address=104.36.82.38:4696 , piece_id=NMDNAT7RVBIA6YQK24Z7XJQSOTIVOBIY5RIHK4EFWJK6N2ZL4UJQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:1JrFvHgEHVLtwK3QYvFTzASCysBCR38qA4Ni8Jvrb1rXUaFy3H, address:104.36.82.38:4696): protocol: rpc: dial tcp 104.36.82.38:4696: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061148564 (202h57m45.792416215s) [noise=true , address=104.36.82.17:4333 , piece_id=AAKCYXF7BIEVWEYXGFSSRQDCQNXLWYBLQLYNX3364RG4656IED7Q, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: rpc: dial tcp 104.36.82.17:4333: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061255446 (202h57m45.792523106s) [noise=true , address=104.36.82.22:4514 , piece_id=73OAAQLS545E64H6VE53NXTAPBX4XJXPZN2EEX53EFUSKIBFTXLQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:104.36.82.22:4514): protocol: rpc: dial tcp 104.36.82.22:4514: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061322663 (202h57m45.792590312s) [noise=true , address=104.36.82.23:4446 , piece_id=VSLMTGTUUZSEP6X7SCPALTJFOA7IJJCIKLJHK6TIYR6VJQLHKTOQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:104.36.82.23:4446): protocol: write tcp 5.161.89.252:28480->104.36.82.23:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061363470 (202h57m45.79263112s) [noise=true , address=104.36.82.2:4762 , piece_id=CMIP2M7AMYAMK7MIDZU5XJRKNVG5ELS7DJBHP73FD2HQHOF2XRTQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12r1mav9KnS7y2V5XxeDexYM4Gv9JcfmqNbML5pwSEyzCZFvJra, address:104.36.82.2:4762): protocol: rpc: dial tcp 104.36.82.2:4762: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061387064 (202h57m45.792654714s) [noise=true , address=104.36.82.5:4864 , piece_id=QU3BHU64HCXUVPBMCEBIYAWX72Q7SGM2I5YF46GNNDM5LJOD542Q, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u, address:104.36.82.5:4864): protocol: rpc: dial tcp 104.36.82.5:4864: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061424123 (202h57m45.792691784s) [noise=true , address=104.36.82.23:4461 , piece_id=JWAHLP4YYBWYQUWEQC5EHJZZZPVYOUTGC2TS5LNXP4SG7EKJFYYQ, seg_pos={0 0}, upload=211167]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:104.36.82.23:4461): protocol: write tcp 5.161.89.252:18768->104.36.82.23:4461: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:24:05.061453258 (202h57m45.792720908s) [seg_pos={0 0}, upload=211167]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:24:05.085107047 (202h57m45.816374697s) [upload=211167]: Done uploading segments... Aug 17 04:24:05.085125321 (202h57m45.816392971s) [upload=211167]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.026504548s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.477370398s FAIL (2025-08-17 04:23:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.477387109s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................X.XX.] duration: 355.443522ms FAIL (2025-08-17 04:23:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 355.459692ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.041449716s FAIL (2025-08-17 04:23:37 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:07.365016247 (202h56m48.096283887s) [upload=211168]: Starting upload Aug 17 04:23:07.365113880 (202h56m48.09638152s) [upload=211168]: Done starting upload Aug 17 04:23:07.365185295 (202h56m48.096452915s) [upload=211168]: Uploading segments... Aug 17 04:23:07.365444052 (202h56m48.096711693s) [upload=211168]: Got next segment. Inline: false Aug 17 04:23:07.365496200 (202h56m48.09676384s) [upload=211168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:07.365506590 (202h56m48.096774229s) [upload=211168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:07.391532593 (202h56m48.122800213s) [seg_pos={0 0}, upload=211168]: Begin upload segment... Aug 17 04:23:07.391561247 (202h56m48.122828867s) [seg_pos={0 0}, upload=211168]: Done begin upload segment. Aug 17 04:23:07.391580673 (202h56m48.122848294s) [noise=true , address=45.91.116.4:4938 , piece_id=F3AVEZ2YIU4EJZCAGSWZGJPNJ7KNSRGRHMNMLPE7XQLMXWGL62XQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.391635377 (202h56m48.122903017s) [noise=true , address=45.91.116.5:4298 , piece_id=SO6VVAFXQBPJMLOQOEFGGC4D6ICGBUK7BVMKHS4D3HYHCQWABJXA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.391895125 (202h56m48.123162745s) [noise=true , address=45.91.116.4:4937 , piece_id=BEYPSNQUECATWBTX4JUM6NP2BBFXQSYVFBNMJFO5O4UNHRLTZLJA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392097867 (202h56m48.123365508s) [noise=true , address=45.91.116.11:4117 , piece_id=OY5KNPDA5EI6OIMR5DBQ7IHSEJQK777NPIKWOZI3RZH7NIF7GWAQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392117784 (202h56m48.123385405s) [noise=true , address=104.36.82.16:4295 , piece_id=7BPNXEZ5JSW4T6DUZED3BAGJZLUAGN2PGDHSXLUV34AC7BM3OGYA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392313993 (202h56m48.123581614s) [noise=true , address=45.91.116.7:4009 , piece_id=36E4LB3QDRVIJ7V3CFTBEU7XVJNMHPXMO3W4U2EWNRRKPQY6RNBQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392395668 (202h56m48.123663308s) [noise=true , address=45.91.116.13:4154 , piece_id=NGDMGFLNRFPCKKQR72KBGJJXOTE5CV2W6OEQAWRJKCTENPUH3T5A, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392493893 (202h56m48.123761513s) [noise=true , address=104.36.82.4:4803 , piece_id=TV3P4HUUHIXIJ7P2ZONBQPAJAY3UZVVLFF4P7TXAZEU47Y6RJ4QQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392681466 (202h56m48.123949106s) [noise=true , address=77.90.31.19:4841 , piece_id=L7TJCCUKL4RISGHZFGOFLKVNKJ3LK2PAAFWQCRJAV7N5VTNPSWDQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392702645 (202h56m48.123970275s) [noise=true , address=104.36.82.12:4151 , piece_id=3Q6OWEBKFSXBCK5AJ6NHFUIO2GXBO3A6QX7X5T2VNQK42KI5KLNQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392977794 (202h56m48.124245414s) [noise=true , address=104.36.82.2:4730 , piece_id=NPIC6AKOB2DBFDFK36KDDJANJZN7JFTISF6DI2K4OTCJIOABPK6Q, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.392988844 (202h56m48.124256475s) [noise=true , address=104.36.82.5:4839 , piece_id=5C6WOWRH4JAPDZFHZYVT7L5GMKU2MYQGWQLIVRUUJJHOOD73HFFA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393185134 (202h56m48.124452754s) [noise=true , address=45.91.116.14:4189 , piece_id=JRIG76HQYCBKFQ67GEXATVH3T5C5I6VLJWNU6XNSKRR5YIAF6UJA, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393273310 (202h56m48.12454094s) [noise=true , address=104.36.82.8:4969 , piece_id=ZP65DV3U7BGQ4OHGXJWB6NSWJBIZIEFLERF7BNBIMTGH5YQWBTNQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393400339 (202h56m48.124667959s) [noise=true , address=45.91.116.10:4081 , piece_id=F4SEIIXTTLFASZHDJBYUBDYLJMAZC6K7MEAIPD3WAQ35UHAEAZMQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.393572042 (202h56m48.124839683s) [noise=true , address=45.91.116.7:4010 , piece_id=6RWG7YB22HSREMJA44RC43SBH3TPQ3F3EDKAC2JULNK2SWSBWNEQ, seg_pos={0 0}, upload=211168]: Uploading piece... Aug 17 04:23:07.461635871 (202h56m48.192903501s) [upload=211168]: Next returned nil segment Aug 17 04:23:07.461646972 (202h56m48.192914602s) [upload=211168]: Waiting for error group managing segments... Aug 17 04:23:07.469880881 (202h56m48.201148501s) [noise=true , address=45.91.116.4:4938 , piece_id=F3AVEZ2YIU4EJZCAGSWZGJPNJ7KNSRGRHMNMLPE7XQLMXWGL62XQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470249144 (202h56m48.201516775s) [noise=true , address=45.91.116.13:4154 , piece_id=NGDMGFLNRFPCKKQR72KBGJJXOTE5CV2W6OEQAWRJKCTENPUH3T5A, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470491772 (202h56m48.201759402s) [noise=true , address=45.91.116.7:4009 , piece_id=36E4LB3QDRVIJ7V3CFTBEU7XVJNMHPXMO3W4U2EWNRRKPQY6RNBQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470580539 (202h56m48.201848169s) [noise=true , address=45.91.116.4:4937 , piece_id=BEYPSNQUECATWBTX4JUM6NP2BBFXQSYVFBNMJFO5O4UNHRLTZLJA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.470934566 (202h56m48.202202196s) [noise=true , address=45.91.116.14:4189 , piece_id=JRIG76HQYCBKFQ67GEXATVH3T5C5I6VLJWNU6XNSKRR5YIAF6UJA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471147235 (202h56m48.202414855s) [noise=true , address=45.91.116.10:4081 , piece_id=F4SEIIXTTLFASZHDJBYUBDYLJMAZC6K7MEAIPD3WAQ35UHAEAZMQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471257373 (202h56m48.202525003s) [noise=true , address=45.91.116.5:4298 , piece_id=SO6VVAFXQBPJMLOQOEFGGC4D6ICGBUK7BVMKHS4D3HYHCQWABJXA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471574630 (202h56m48.20284225s) [noise=true , address=45.91.116.7:4010 , piece_id=6RWG7YB22HSREMJA44RC43SBH3TPQ3F3EDKAC2JULNK2SWSBWNEQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.471656044 (202h56m48.202923674s) [noise=true , address=45.91.116.11:4117 , piece_id=OY5KNPDA5EI6OIMR5DBQ7IHSEJQK777NPIKWOZI3RZH7NIF7GWAQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:07.472781833 (202h56m48.204049453s) [noise=true , address=77.90.31.19:4841 , piece_id=L7TJCCUKL4RISGHZFGOFLKVNKJ3LK2PAAFWQCRJAV7N5VTNPSWDQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: <nil> Aug 17 04:23:37.353223355 (202h57m18.084491014s) [noise=true , address=104.36.82.4:4803 , piece_id=TV3P4HUUHIXIJ7P2ZONBQPAJAY3UZVVLFF4P7TXAZEU47Y6RJ4QQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: rpc: dial tcp 104.36.82.4:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353345955 (202h57m18.084613605s) [noise=true , address=104.36.82.5:4839 , piece_id=5C6WOWRH4JAPDZFHZYVT7L5GMKU2MYQGWQLIVRUUJJHOOD73HFFA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: rpc: dial tcp 104.36.82.5:4839: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353365382 (202h57m18.084633042s) [noise=true , address=104.36.82.12:4151 , piece_id=3Q6OWEBKFSXBCK5AJ6NHFUIO2GXBO3A6QX7X5T2VNQK42KI5KLNQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: rpc: dial tcp 104.36.82.12:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353416519 (202h57m18.084684169s) [noise=true , address=104.36.82.2:4730 , piece_id=NPIC6AKOB2DBFDFK36KDDJANJZN7JFTISF6DI2K4OTCJIOABPK6Q, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: rpc: dial tcp 104.36.82.2:4730: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353477142 (202h57m18.084744793s) [noise=true , address=104.36.82.16:4295 , piece_id=7BPNXEZ5JSW4T6DUZED3BAGJZLUAGN2PGDHSXLUV34AC7BM3OGYA, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: write tcp 5.161.89.252:26914->104.36.82.16:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353487252 (202h57m18.084754901s) [noise=true , address=104.36.82.8:4969 , piece_id=ZP65DV3U7BGQ4OHGXJWB6NSWJBIZIEFLERF7BNBIMTGH5YQWBTNQ, seg_pos={0 0}, upload=211168]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: rpc: dial tcp 104.36.82.8:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:37.353530393 (202h57m18.084798093s) [seg_pos={0 0}, upload=211168]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:37.392615539 (202h57m18.123883199s) [upload=211168]: Done uploading segments... Aug 17 04:23:37.392649854 (202h57m18.123917504s) [upload=211168]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.041464353s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.021963799s FAIL (2025-08-17 04:23:20 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== RZBUDJSRWINPUWSC7GFCNZHLNEHAXEGHRPHWCXSDXOQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.659271235 (202h56m1.390538885s) [noise=true , address=77.90.31.11:4575 , piece_id=LUZIVRMU4TBV6EDNQC7OIZPYXJPUGA4JCHCMCFDXVF4ON4VDZSYQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.659762459 (202h56m1.391030109s) [noise=true , address=104.36.82.23:4450 , piece_id=OSPHVCAOCPNJCZOLF2HB6E4OAOJRFBVPGAUDN5NQ5QEV4VA4CX6Q, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.659882656 (202h56m1.391150306s) [noise=true , address=104.36.82.22:4525 , piece_id=5C4FM35WLBKV65B5HKSEOGF42PZF2XVMJQOBMS4FNUS4OOGXI5QQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.660221423 (202h56m1.391489073s) [noise=true , address=45.91.116.9:4275 , piece_id=3U6HDGYNASA55JH2LOULFRV5SL4RJ7LLAYGNQOESDUWP66RX3ABA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.660337121 (202h56m1.391604781s) [noise=true , address=79.127.205.233:4624 , piece_id=VBFMBRIYT5QUBQU6MLS3OTETOG5EQD4KTP7WNNWIGWQD7XKYRPSA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.660648777 (202h56m1.391916428s) [noise=true , address=104.36.82.5:4868 , piece_id=VLZRKZ564XA52UQLUF6GQKWZKURNFJ2KBRYV5PJZVESYKV33TULA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.660707408 (202h56m1.391975059s) [noise=true , address=104.36.82.60:4359 , piece_id=5NLPTKBQWPBQKQ7DMDDVE4RVXK6OSLM5LEEZXLOGVXJYJRTKSYXQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.661069400 (202h56m1.392337049s) [noise=true , address=66.23.193.175:4560 , piece_id=NEVOPRSGTTMOEOLC4ZNK6UQ6OKATBYEDTQTJDH7OHX7IM2QC35CA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.661135154 (202h56m1.392402814s) [noise=true , address=66.23.193.175:4556 , piece_id=G2TLBNE2RLFO2MP63GDQWT2GYOKHKUMCQCG6ZQV2LHNK3NOIIN5A, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.661541489 (202h56m1.392809139s) [noise=true , address=104.36.82.21:4506 , piece_id=2YUYH6CBMVXSUJYZSWF4QPFMMC27FEQ2OUFGKUQXUGR3HVRPNEBQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.661551668 (202h56m1.392819318s) [noise=true , address=77.90.31.13:4629 , piece_id=UBMBGMWPDRL6UROGCIKFM5VYKXV27OF3GBDVSEMT4QJ3LGMSH5XA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.661987599 (202h56m1.393255248s) [noise=true , address=79.127.201.215:4355 , piece_id=PJ7QBB4GMADCRJYCMKMRI6CR3JASCHWNB4LU4IT6UVJE2P6C5JXQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.662377022 (202h56m1.393644672s) [noise=true , address=104.36.82.3:4789 , piece_id=OH7QNAFKWXVDZ4TJD56G4TFJQ64WOUHENRM6BTCHAIIREEFSR6UA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.662846836 (202h56m1.394114486s) [noise=true , address=77.90.31.10:4543 , piece_id=GUU6FAVIN2VPFQM2TG5J6IJQENHU5YOIL6TPCJXW5Y6EF3X6IIPQ, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.663252920 (202h56m1.394520571s) [noise=true , address=77.90.31.15:4706 , piece_id=U3S2IJ5N6HIMIM3JU5BRPHEN6MV2GDEXNYDBKC2ZIFGOHHG5O44A, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.665141165 (202h56m1.396408815s) [noise=true , address=77.90.31.19:4859 , piece_id=VWQR5ORINWZKM6IVWZQDGZEXLDSQ4AWYQLCIOMYLYJNIZ6CBSENA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.665622090 (202h56m1.39688975s) [noise=true , address=104.36.82.22:4521 , piece_id=CBMSB5TKD5QBDG3SRGR36P53AOKTOIYMNY5R4IK7UVH274KWXFRA, seg_pos={0 0}, upload=211156]: Uploading piece... Aug 17 04:22:20.768112265 (202h56m1.499379915s) [upload=211156]: Next returned nil segment Aug 17 04:22:20.768130330 (202h56m1.49939798s) [upload=211156]: Waiting for error group managing segments... Aug 17 04:22:20.777159858 (202h56m1.508427518s) [noise=true , address=45.91.116.9:4275 , piece_id=3U6HDGYNASA55JH2LOULFRV5SL4RJ7LLAYGNQOESDUWP66RX3ABA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:20.778285516 (202h56m1.509553166s) [noise=true , address=77.90.31.19:4859 , piece_id=VWQR5ORINWZKM6IVWZQDGZEXLDSQ4AWYQLCIOMYLYJNIZ6CBSENA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:20.778578577 (202h56m1.509846228s) [noise=true , address=77.90.31.11:4575 , piece_id=LUZIVRMU4TBV6EDNQC7OIZPYXJPUGA4JCHCMCFDXVF4ON4VDZSYQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:20.814497535 (202h56m1.545765194s) [noise=true , address=77.90.31.10:4543 , piece_id=GUU6FAVIN2VPFQM2TG5J6IJQENHU5YOIL6TPCJXW5Y6EF3X6IIPQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:20.819093106 (202h56m1.550360736s) [noise=true , address=77.90.31.15:4706 , piece_id=U3S2IJ5N6HIMIM3JU5BRPHEN6MV2GDEXNYDBKC2ZIFGOHHG5O44A, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:20.843970876 (202h56m1.575238537s) [noise=true , address=77.90.31.13:4629 , piece_id=UBMBGMWPDRL6UROGCIKFM5VYKXV27OF3GBDVSEMT4QJ3LGMSH5XA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:21.076022838 (202h56m1.807290498s) [noise=true , address=79.127.201.215:4355 , piece_id=PJ7QBB4GMADCRJYCMKMRI6CR3JASCHWNB4LU4IT6UVJE2P6C5JXQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:21.184042963 (202h56m1.915310593s) [noise=true , address=79.127.205.233:4624 , piece_id=VBFMBRIYT5QUBQU6MLS3OTETOG5EQD4KTP7WNNWIGWQD7XKYRPSA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:21.259522661 (202h56m1.990790281s) [noise=true , address=66.23.193.175:4560 , piece_id=NEVOPRSGTTMOEOLC4ZNK6UQ6OKATBYEDTQTJDH7OHX7IM2QC35CA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:22:21.259777681 (202h56m1.991045311s) [noise=true , address=66.23.193.175:4556 , piece_id=G2TLBNE2RLFO2MP63GDQWT2GYOKHKUMCQCG6ZQV2LHNK3NOIIN5A, seg_pos={0 0}, upload=211156]: Done uploading piece. err: <nil> Aug 17 04:23:20.617171186 (202h57m1.348438806s) [noise=true , address=104.36.82.44:4890 , piece_id=KJTXJ2ZI5RZBUDJSRWINPUWSC7GFCNZHLNEHAXEGHRPHWCXSDXOQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:1XYhMcqH63UiHYE4ofxDxjXz2HMvN1RfQjgxxxHcUst7P3aN7M, address:104.36.82.44:4890): protocol: write tcp 5.161.89.252:60690->104.36.82.44:4890: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617223303 (202h57m1.348490923s) [noise=true , address=104.36.82.5:4868 , piece_id=VLZRKZ564XA52UQLUF6GQKWZKURNFJ2KBRYV5PJZVESYKV33TULA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v, address:104.36.82.5:4868): protocol: rpc: dial tcp 104.36.82.5:4868: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617253381 (202h57m1.348521011s) [noise=true , address=104.36.82.22:4521 , piece_id=CBMSB5TKD5QBDG3SRGR36P53AOKTOIYMNY5R4IK7UVH274KWXFRA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:1zWiwpaV58h8YKsAgw24fUWHBBYYyqF4UX6GJV316Uxo9Y7gt1, address:104.36.82.22:4521): protocol: rpc: dial tcp 104.36.82.22:4521: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617262628 (202h57m1.348530248s) [noise=true , address=104.36.82.60:4359 , piece_id=5NLPTKBQWPBQKQ7DMDDVE4RVXK6OSLM5LEEZXLOGVXJYJRTKSYXQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:1YgFc7P9RCbYw9u2wfow8Z3DMwFvJ6AnCsvzoxU9pTwPWGw8MV, address:104.36.82.60:4359): protocol: rpc: dial tcp 104.36.82.60:4359: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617292925 (202h57m1.348560545s) [noise=true , address=104.36.82.23:4450 , piece_id=OSPHVCAOCPNJCZOLF2HB6E4OAOJRFBVPGAUDN5NQ5QEV4VA4CX6Q, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms, address:104.36.82.23:4450): protocol: rpc: dial tcp 104.36.82.23:4450: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617302272 (202h57m1.348569893s) [noise=true , address=104.36.82.21:4506 , piece_id=2YUYH6CBMVXSUJYZSWF4QPFMMC27FEQ2OUFGKUQXUGR3HVRPNEBQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:12YUTchJGJE2EGXqiGZEcLjQmFQCazjV8Ahi8ogJhuqTE4H6n7T, address:104.36.82.21:4506): protocol: rpc: dial tcp 104.36.82.21:4506: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617330305 (202h57m1.348597925s) [noise=true , address=104.36.82.22:4525 , piece_id=5C4FM35WLBKV65B5HKSEOGF42PZF2XVMJQOBMS4FNUS4OOGXI5QQ, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:12Yx3qg5NAW8iAw56mkwALjkkhyFF5feKgTZxcTG31RxBsubh5A, address:104.36.82.22:4525): protocol: rpc: dial tcp 104.36.82.22:4525: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617339573 (202h57m1.348607192s) [noise=true , address=104.36.82.3:4789 , piece_id=OH7QNAFKWXVDZ4TJD56G4TFJQ64WOUHENRM6BTCHAIIREEFSR6UA, seg_pos={0 0}, upload=211156]: Done uploading piece. err: ecclient: upload failed (node:12eQnnZm3aoAEA6cHJshERFmZuyh5cUjqzYK687CS27CQ8nGzrT, address:104.36.82.3:4789): protocol: write tcp 5.161.89.252:20166->104.36.82.3:4789: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:20.617359891 (202h57m1.348627521s) [seg_pos={0 0}, upload=211156]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:20.637619750 (202h57m1.36888738s) [upload=211156]: Done uploading segments... Aug 17 04:23:20.637638765 (202h57m1.368906395s) [upload=211156]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.021976775s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.592635731s FAIL (2025-08-17 04:23:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.592650429s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.016101259s FAIL (2025-08-17 04:23:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:22:37.282136699 (202h56m18.013404359s) [upload=211161]: Starting upload Aug 17 04:22:37.282266875 (202h56m18.013534534s) [upload=211161]: Done starting upload Aug 17 04:22:37.282424311 (202h56m18.013691971s) [upload=211161]: Uploading segments... Aug 17 04:22:37.282661748 (202h56m18.013929398s) [upload=211161]: Got next segment. Inline: false Aug 17 04:22:37.282746938 (202h56m18.014014588s) [upload=211161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:37.282753451 (202h56m18.014021111s) [upload=211161]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:37.306766945 (202h56m18.038034596s) [seg_pos={0 0}, upload=211161]: Begin upload segment... Aug 17 04:22:37.306829533 (202h56m18.038097183s) [seg_pos={0 0}, upload=211161]: Done begin upload segment. Aug 17 04:22:37.306857777 (202h56m18.038125427s) [noise=true , address=45.91.116.7:4010 , piece_id=6TZQ7K5GW3LEZ67BL6CLARANRMO4DLWI3GPRCMLFADMR2PGAW2TA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.306872154 (202h56m18.038139804s) [noise=true , address=77.90.31.12:4592 , piece_id=4WU7FLO7XJFSO5CZGBC7NEBLISHPSYSTX5CVTY2GE6GX2DKVDQ6A, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.307368036 (202h56m18.038635697s) [noise=true , address=45.91.116.5:4297 , piece_id=GDO4AAAB66DEHPG47SQKVZYIMG63BV3QLRU7EGPR6KRBZTODZPGQ, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.307495737 (202h56m18.038763387s) [noise=true , address=104.36.82.2:4730 , piece_id=YZIY7FTVGCWZO3EAXDAEMV7FM4EZIAKBGXOROK76FHLUEZ2XJ3YA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.307831539 (202h56m18.039099189s) [noise=true , address=45.91.116.6:4973 , piece_id=L4WPA5DK5HBZXOUASPQY36ZYOGGOFHZIGXJWRW5RR5Z5U7GABUGQ, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.308132115 (202h56m18.039399776s) [noise=true , address=77.90.31.14:4661 , piece_id=HH6EM7TTHQZF2TX2DI7YWY7NIBAGM2J4RN45SV4VLXCRRWJSH6KQ, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.308245488 (202h56m18.039513148s) [noise=true , address=45.91.116.12:4227 , piece_id=HKT2MI4GWNR3FECTR5O42X4P7H3GDFI37DZ3APELD22ZOKVLTNKA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.308643939 (202h56m18.039911588s) [noise=true , address=104.36.82.16:4294 , piece_id=AFPSOBLP6GQLMJEXNV7UORIABKLWW6FOYQGHJBOXPEASGSMTVQXQ, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.308660710 (202h56m18.03992836s) [noise=true , address=77.90.31.11:4553 , piece_id=VIGW7JIFP6ZHVP3PWC2YPMIIFX6F3NJSDCCPYHZMRO6J66XMEWHA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.309047369 (202h56m18.040315018s) [noise=true , address=104.36.82.6:4873 , piece_id=SLJBU7RHD5PBJBGKIL4ATT77K2LM24M3MCLPXW2QR6QSMLHGI4TA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.309362421 (202h56m18.040630072s) [noise=true , address=45.91.116.5:4298 , piece_id=5NMQULJJCY7REDEOOO3AKSZGAKJDO4DUO35SE5FN33LNHVN6CC5Q, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.309426511 (202h56m18.040694162s) [noise=true , address=77.90.31.11:4554 , piece_id=D6FW5TAR3GRD4O3KTKKDHQOGF3OEIKCNWBQVFHXFE5AJGVAUZKBA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.309918368 (202h56m18.041186018s) [noise=true , address=45.91.116.11:4118 , piece_id=BKVS7HLUTGV2WCBJII2GGLFQJEM2XZCZ4MJOMOIJQQ64BESEYUDQ, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.310087315 (202h56m18.041354966s) [noise=true , address=104.36.82.4:4803 , piece_id=B7GTZBDHT4TLNSNU3VWOMOOVDSPPQC5OQTH7GAWTENFLKWAPJ7RA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.310328289 (202h56m18.04159594s) [noise=true , address=104.36.82.5:4840 , piece_id=HMFKOYMFOAZ3PISDXG35PNQJKWXGHOM3VE46HGRWMRFSQVWZYL3A, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.310610461 (202h56m18.041878111s) [noise=true , address=104.36.82.10:4078 , piece_id=IQWNKHNIKY5U4GJ4C6OBBSWMSDCB3V6SKMNFQ6CVGBUAMIPO3WZA, seg_pos={0 0}, upload=211161]: Uploading piece... Aug 17 04:22:37.415752878 (202h56m18.147020538s) [upload=211161]: Next returned nil segment Aug 17 04:22:37.415768968 (202h56m18.147036619s) [upload=211161]: Waiting for error group managing segments... Aug 17 04:22:37.423737699 (202h56m18.15500535s) [noise=true , address=45.91.116.7:4010 , piece_id=6TZQ7K5GW3LEZ67BL6CLARANRMO4DLWI3GPRCMLFADMR2PGAW2TA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.424109329 (202h56m18.155376979s) [noise=true , address=45.91.116.11:4118 , piece_id=BKVS7HLUTGV2WCBJII2GGLFQJEM2XZCZ4MJOMOIJQQ64BESEYUDQ, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.425128797 (202h56m18.156396448s) [noise=true , address=77.90.31.11:4554 , piece_id=D6FW5TAR3GRD4O3KTKKDHQOGF3OEIKCNWBQVFHXFE5AJGVAUZKBA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.425590578 (202h56m18.156858228s) [noise=true , address=45.91.116.5:4297 , piece_id=GDO4AAAB66DEHPG47SQKVZYIMG63BV3QLRU7EGPR6KRBZTODZPGQ, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.425740620 (202h56m18.157008271s) [noise=true , address=45.91.116.6:4973 , piece_id=L4WPA5DK5HBZXOUASPQY36ZYOGGOFHZIGXJWRW5RR5Z5U7GABUGQ, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.426306626 (202h56m18.157574276s) [noise=true , address=45.91.116.12:4227 , piece_id=HKT2MI4GWNR3FECTR5O42X4P7H3GDFI37DZ3APELD22ZOKVLTNKA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.427437304 (202h56m18.158704954s) [noise=true , address=77.90.31.11:4553 , piece_id=VIGW7JIFP6ZHVP3PWC2YPMIIFX6F3NJSDCCPYHZMRO6J66XMEWHA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.428148162 (202h56m18.159415813s) [noise=true , address=45.91.116.5:4298 , piece_id=5NMQULJJCY7REDEOOO3AKSZGAKJDO4DUO35SE5FN33LNHVN6CC5Q, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.428751969 (202h56m18.160019619s) [noise=true , address=77.90.31.14:4661 , piece_id=HH6EM7TTHQZF2TX2DI7YWY7NIBAGM2J4RN45SV4VLXCRRWJSH6KQ, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:22:37.549945763 (202h56m18.281213424s) [noise=true , address=77.90.31.12:4592 , piece_id=4WU7FLO7XJFSO5CZGBC7NEBLISHPSYSTX5CVTY2GE6GX2DKVDQ6A, seg_pos={0 0}, upload=211161]: Done uploading piece. err: <nil> Aug 17 04:23:07.268697549 (202h56m47.999965199s) [noise=true , address=104.36.82.16:4294 , piece_id=AFPSOBLP6GQLMJEXNV7UORIABKLWW6FOYQGHJBOXPEASGSMTVQXQ, seg_pos={0 0}, upload=211161]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: write tcp 5.161.89.252:33148->104.36.82.16:4294: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:07.268761720 (202h56m48.00002937s) [noise=true , address=104.36.82.6:4873 , piece_id=SLJBU7RHD5PBJBGKIL4ATT77K2LM24M3MCLPXW2QR6QSMLHGI4TA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: rpc: dial tcp 104.36.82.6:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:07.268836390 (202h56m48.000104041s) [noise=true , address=104.36.82.5:4840 , piece_id=HMFKOYMFOAZ3PISDXG35PNQJKWXGHOM3VE46HGRWMRFSQVWZYL3A, seg_pos={0 0}, upload=211161]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: rpc: dial tcp 104.36.82.5:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:07.268847943 (202h56m48.000115592s) [noise=true , address=104.36.82.4:4803 , piece_id=B7GTZBDHT4TLNSNU3VWOMOOVDSPPQC5OQTH7GAWTENFLKWAPJ7RA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: rpc: dial tcp 104.36.82.4:4803: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:07.268879923 (202h56m48.000147572s) [noise=true , address=104.36.82.10:4078 , piece_id=IQWNKHNIKY5U4GJ4C6OBBSWMSDCB3V6SKMNFQ6CVGBUAMIPO3WZA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: rpc: dial tcp 104.36.82.10:4078: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:07.268913426 (202h56m48.000181065s) [noise=true , address=104.36.82.2:4730 , piece_id=YZIY7FTVGCWZO3EAXDAEMV7FM4EZIAKBGXOROK76FHLUEZ2XJ3YA, seg_pos={0 0}, upload=211161]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:104.36.82.2:4730): protocol: write tcp 5.161.89.252:33160->104.36.82.2:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:07.268964141 (202h56m48.000231781s) [seg_pos={0 0}, upload=211161]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:07.283404084 (202h56m48.014671735s) [upload=211161]: Done uploading segments... Aug 17 04:23:07.283417990 (202h56m48.014685641s) [upload=211161]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.016122188s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000954981s FAIL (2025-08-17 04:23:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000974637s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.023360368s FAIL (2025-08-17 04:23:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== Aug 17 04:22:04.980426010 (202h55m45.71169366s) [upload=211152]: Starting upload Aug 17 04:22:04.980493026 (202h55m45.711760686s) [upload=211152]: Done starting upload Aug 17 04:22:04.980622350 (202h55m45.71189002s) [upload=211152]: Uploading segments... Aug 17 04:22:04.980871900 (202h55m45.71213955s) [upload=211152]: Got next segment. Inline: false Aug 17 04:22:04.980918518 (202h55m45.712186167s) [upload=211152]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:22:04.980924318 (202h55m45.712191968s) [upload=211152]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:22:05.009416233 (202h55m45.740683884s) [seg_pos={0 0}, upload=211152]: Begin upload segment... Aug 17 04:22:05.009477709 (202h55m45.740745379s) [seg_pos={0 0}, upload=211152]: Done begin upload segment. Aug 17 04:22:05.009506133 (202h55m45.740773863s) [noise=true , address=104.36.82.22:4525 , piece_id=KN7RZCNODO475TTR3YVOZYEZL2TMFSKF6SIZHRIWPK5G7ONIUCNQ, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.009531641 (202h55m45.740799292s) [noise=true , address=77.90.31.21:4902 , piece_id=BWNGF7PL6I2CSUMKJLA7TQAKABYRF55V3MTSP4ODUHXRR7EDQ7NA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.010102216 (202h55m45.741369865s) [noise=true , address=79.127.201.215:4353 , piece_id=Q3VLBEADD5U54FHBIRY5NC3LHTRRTKIWHNWJQHOYSMBN5DLERLZA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.010191964 (202h55m45.741459614s) [noise=true , address=104.36.82.59:4309 , piece_id=Z4LU5CXOBFHFHJDZL2L5DJWBUIQXMPN6CY6LGXH2A7PK4JBQCGUA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.010518007 (202h55m45.741785657s) [noise=true , address=77.90.31.14:4695 , piece_id=FG36MMGXR2YYFTKI5WOJ2GE7OAADCOCRE7UH2PTYR7OS7BRPL4NQ, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.011000877 (202h55m45.742268537s) [noise=true , address=77.90.31.12:4602 , piece_id=O4CXBL5UCYYWZM4PD4KKVOVIVNZXFOQY3GO636DJXRP6XI4T55IA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.011588333 (202h55m45.742855983s) [noise=true , address=104.36.82.60:4360 , piece_id=FQEEA4R7JJ7QM6WBL6DLRRZ6XF42HTW4EHRSW3NALGZBP37ODWLQ, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.012142886 (202h55m45.743410537s) [noise=true , address=104.36.82.17:4351 , piece_id=YLEO4LVCOOFYW2XHRXA7ELKMOY3SWRSK32KRVUJ5SUF3YYEGNV7A, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.012588816 (202h55m45.743856466s) [noise=true , address=104.36.82.4:4835 , piece_id=TP2MMTT57JSLCHXGHVBLEN3OU7RRA3OKMTI44EBXYGYWO3GU3NDQ, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.013006663 (202h55m45.744274313s) [noise=true , address=45.91.116.7:4023 , piece_id=7GRFH7BZJH2VW6TMLUMBNQETWTJ4X7QJCKCYQTXS4E4YVTDHRH7Q, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.013402709 (202h55m45.744670358s) [noise=true , address=104.36.82.40:4967 , piece_id=N7RNXTWTA5U6PVMOJE5ILYVY6ACLRAZ3MDJ3VR4LXSGVKDGH5MAA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.013819201 (202h55m45.745086852s) [noise=true , address=79.127.205.246:4288 , piece_id=EDZOBXEZSGS2LL5QQKBVKXK6CNHSVAMTDDMVA326BKOMWRR3TRKA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.014197504 (202h55m45.745465154s) [noise=true , address=104.36.82.44:4891 , piece_id=HQPIT6EDLWEWCIAU4W2VBYQZBC5BVCLXSXSFJ24C7RLVAOVXNLWA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.014635208 (202h55m45.745902868s) [noise=true , address=104.36.82.40:4962 , piece_id=NOK4JQREC3I5FZ3ZEWRHUIEQGC5GRT63KMGZERTMYKBRUOVJFVSQ, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.015185594 (202h55m45.746453244s) [noise=true , address=79.127.201.214:4320 , piece_id=LRSYRNPBDPRPT4WX4VPGKUBAANSKKHB4XFEBECO4KZLLH2TFNM7A, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.017729183 (202h55m45.748996843s) [noise=true , address=104.36.82.58:4297 , piece_id=H4QUJL3CODU2J3CZ5GZ4WLH5BAXLUA4CEZS62GNQWJ4BRN452GQQ, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.018407158 (202h55m45.749674808s) [noise=true , address=104.36.82.34:4564 , piece_id=5GOSV2RCHPKZLRLFLEXREJUIEJ4JS3USAZT5WNO5P5LBS36FYMNA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.018982852 (202h55m45.750250502s) [noise=true , address=104.36.82.23:4452 , piece_id=K7ILSYBKBBP4AWWRS4ON4CMKXNW6B3TJOGE6B2EBETPEH6MXAUKA, seg_pos={0 0}, upload=211152]: Uploading piece... Aug 17 04:22:05.111470348 (202h55m45.842737988s) [upload=211152]: Next returned nil segment Aug 17 04:22:05.111485156 (202h55m45.842752796s) [upload=211152]: Waiting for error group managing segments... Aug 17 04:22:05.141273040 (202h55m45.87254069s) [noise=true , address=45.91.116.7:4023 , piece_id=7GRFH7BZJH2VW6TMLUMBNQETWTJ4X7QJCKCYQTXS4E4YVTDHRH7Q, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:22:05.172953948 (202h55m45.904221598s) [noise=true , address=77.90.31.14:4695 , piece_id=FG36MMGXR2YYFTKI5WOJ2GE7OAADCOCRE7UH2PTYR7OS7BRPL4NQ, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:22:05.197993424 (202h55m45.929261075s) [noise=true , address=77.90.31.21:4902 , piece_id=BWNGF7PL6I2CSUMKJLA7TQAKABYRF55V3MTSP4ODUHXRR7EDQ7NA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:22:05.372436719 (202h55m46.103704369s) [noise=true , address=79.127.201.214:4320 , piece_id=LRSYRNPBDPRPT4WX4VPGKUBAANSKKHB4XFEBECO4KZLLH2TFNM7A, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:22:05.388882378 (202h55m46.120150027s) [noise=true , address=79.127.201.215:4353 , piece_id=Q3VLBEADD5U54FHBIRY5NC3LHTRRTKIWHNWJQHOYSMBN5DLERLZA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:22:05.543028073 (202h55m46.274295722s) [noise=true , address=79.127.205.246:4288 , piece_id=EDZOBXEZSGS2LL5QQKBVKXK6CNHSVAMTDDMVA326BKOMWRR3TRKA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:22:05.585577189 (202h55m46.316844839s) [noise=true , address=77.90.31.12:4602 , piece_id=O4CXBL5UCYYWZM4PD4KKVOVIVNZXFOQY3GO636DJXRP6XI4T55IA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: <nil> Aug 17 04:23:04.965801364 (202h56m45.697068994s) [noise=true , address=104.36.82.59:4309 , piece_id=Z4LU5CXOBFHFHJDZL2L5DJWBUIQXMPN6CY6LGXH2A7PK4JBQCGUA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:12txGN8RDcyMycMhkRtdZAUkMK9L2ZnqPihEGZ2YNXJvu89RUU1, address:104.36.82.59:4309): protocol: rpc: dial tcp 104.36.82.59:4309: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.965853041 (202h56m45.697120661s) [noise=true , address=104.36.82.17:4351 , piece_id=YLEO4LVCOOFYW2XHRXA7ELKMOY3SWRSK32KRVUJ5SUF3YYEGNV7A, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:17bJjsLQc1F8ARek42WZXsJgNPtbzGn3Badv7X9UhD8auyBP3j, address:104.36.82.17:4351): protocol: rpc: dial tcp 104.36.82.17:4351: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.965886905 (202h56m45.697154535s) [noise=true , address=104.36.82.44:4891 , piece_id=HQPIT6EDLWEWCIAU4W2VBYQZBC5BVCLXSXSFJ24C7RLVAOVXNLWA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:1RHLM4MJmtKoHsUnsqccy1ehyD7zVSn6gnJNp7yJJmZhTJqPkU, address:104.36.82.44:4891): protocol: rpc: dial tcp 104.36.82.44:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.965917362 (202h56m45.697184982s) [noise=true , address=104.36.82.22:4525 , piece_id=KN7RZCNODO475TTR3YVOZYEZL2TMFSKF6SIZHRIWPK5G7ONIUCNQ, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:12Yx3qg5NAW8iAw56mkwALjkkhyFF5feKgTZxcTG31RxBsubh5A, address:104.36.82.22:4525): protocol: rpc: dial tcp 104.36.82.22:4525: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.965952157 (202h56m45.697219787s) [noise=true , address=104.36.82.40:4967 , piece_id=N7RNXTWTA5U6PVMOJE5ILYVY6ACLRAZ3MDJ3VR4LXSGVKDGH5MAA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:1Zz78xER2nL3XBQemnLxWTPUxtVySJv9yK2e7FfUcNeg5XK1r4, address:104.36.82.40:4967): protocol: rpc: dial tcp 104.36.82.40:4967: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966039993 (202h56m45.697307613s) [noise=true , address=104.36.82.4:4835 , piece_id=TP2MMTT57JSLCHXGHVBLEN3OU7RRA3OKMTI44EBXYGYWO3GU3NDQ, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:12WYQwhW2KDjDyweEx5FLYRkGwvW15f6eVHzbdrysiHeuKNtaoN, address:104.36.82.4:4835): protocol: rpc: dial tcp 104.36.82.4:4835: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966060973 (202h56m45.697328603s) [noise=true , address=104.36.82.60:4360 , piece_id=FQEEA4R7JJ7QM6WBL6DLRRZ6XF42HTW4EHRSW3NALGZBP37ODWLQ, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk, address:104.36.82.60:4360): protocol: rpc: dial tcp 104.36.82.60:4360: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966090848 (202h56m45.697358468s) [noise=true , address=104.36.82.23:4452 , piece_id=K7ILSYBKBBP4AWWRS4ON4CMKXNW6B3TJOGE6B2EBETPEH6MXAUKA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:1sAh1mLmzbtmc4o2FAZssW8Y7iDDNmNTCXdCS8xrH8PBVQqxG5, address:104.36.82.23:4452): protocol: rpc: dial tcp 104.36.82.23:4452: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966113992 (202h56m45.697381622s) [noise=true , address=104.36.82.40:4962 , piece_id=NOK4JQREC3I5FZ3ZEWRHUIEQGC5GRT63KMGZERTMYKBRUOVJFVSQ, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:13va7LjDtQV2ob7mrUaJUXc64HmZG3svoDaErmypjYHVAcw1o3, address:104.36.82.40:4962): protocol: rpc: dial tcp 104.36.82.40:4962: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966121305 (202h56m45.697388926s) [noise=true , address=104.36.82.34:4564 , piece_id=5GOSV2RCHPKZLRLFLEXREJUIEJ4JS3USAZT5WNO5P5LBS36FYMNA, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:121WW4uBQe8bGQKkasQQKT3k4EAdzncsG1ebNvgKf8S5SyNttCc, address:104.36.82.34:4564): protocol: write tcp 5.161.89.252:19088->104.36.82.34:4564: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966134250 (202h56m45.69740186s) [noise=true , address=104.36.82.58:4297 , piece_id=H4QUJL3CODU2J3CZ5GZ4WLH5BAXLUA4CEZS62GNQWJ4BRN452GQQ, seg_pos={0 0}, upload=211152]: Done uploading piece. err: ecclient: upload failed (node:12fVAGmLTkEZ2oauRnaMgyV9vvbRKZHj6eKW5vf2yZbPMDpRwfW, address:104.36.82.58:4297): protocol: rpc: dial tcp 104.36.82.58:4297: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:04.966153927 (202h56m45.697421548s) [seg_pos={0 0}, upload=211152]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Aug 17 04:23:04.987613785 (202h56m45.718881404s) [upload=211152]: Done uploading segments... Aug 17 04:23:04.987622972 (202h56m45.718890592s) [upload=211152]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.023380095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 30.000913054s FAIL (2025-08-17 04:22:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00093254s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.016676612s FAIL (2025-08-17 04:22:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== JZLH6LYYHFQHRGYOF534T4VXU4LPEQKOLPJJ4B7ZMKA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589186494 (202h55m1.320454124s) [noise=true , address=104.36.82.23:4474 , piece_id=NOKOTMAIA5JSXOL7AB6I5EOMNSRY7GTPNSXJAU3BIJNLTVNZPPHA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589218755 (202h55m1.320486385s) [noise=true , address=45.91.116.13:4184 , piece_id=EK5KUV3VCESUPXP7IPKYFN72X624GLPXKVJI3JNZCHNDFFPK3IXQ, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589487070 (202h55m1.3207547s) [noise=true , address=104.36.82.35:4612 , piece_id=4HEDST6C47ADCLUQ2B7IT3VVTDKZJ3ZR3TGXVX63SRZ34T5BTRXA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589495095 (202h55m1.320762725s) [noise=true , address=104.36.82.8:4992 , piece_id=UCOJ2KSSCBVEGVZGBVA7ISTPY4AFLP5XZYF42WM4K26VTS3WHWZA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589699320 (202h55m1.32096695s) [noise=true , address=66.23.193.175:4553 , piece_id=4WTQ7KDTUUYJTYBAPDMMJTMS36BHXGWDUCID7ZFSIISJUNVIEPPQ, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589743252 (202h55m1.321010883s) [noise=true , address=104.36.82.44:4883 , piece_id=ZDK7UCS365YHVABBUARN7TTEJTHW5YX42KR7D5E6DS53FDGOQ66Q, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.589972623 (202h55m1.321240253s) [noise=true , address=104.36.82.21:4508 , piece_id=RITMU5RREPWJKX35AG3E5C4QNZBPASKUMKPBOEP5N7WAL7UGBNUA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.590186106 (202h55m1.321453736s) [noise=true , address=104.36.82.60:4360 , piece_id=76KZRVKEGTCRZKLI2IH6YUNGCNB64SNWBTBRHZCLZWWH7ZQVK46Q, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.590392795 (202h55m1.321660415s) [noise=true , address=104.36.82.34:4557 , piece_id=AQPVQH7IKFYOVO3SY4JB3IQWEHRGTNQYQRZMFPHY4WZ7B336X7KA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.590635592 (202h55m1.321903212s) [noise=true , address=77.90.31.12:4621 , piece_id=JQ7M5YUHU5NVM6BF5EG2RJKQKMWYL2WIJ446DDXDJCRSMRXON27A, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.590861127 (202h55m1.322128757s) [noise=true , address=104.36.82.16:4308 , piece_id=A6TQMADKLCOUU7HW36XCG4XMTPVG4DZXWOYN2HF5B2V6XXBSB5TQ, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.591067986 (202h55m1.322335616s) [noise=true , address=77.90.31.19:4872 , piece_id=CLAAH6J3YDZ3TJWF5YKKYM64IWJFV5ZZVY6QE4JO33G3DFF3REVA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.591289043 (202h55m1.322556673s) [noise=true , address=45.91.116.6:4981 , piece_id=M3INZ2P4EMIHIN25MDOS6DZYOB2LLX4EEYP3LZXYWRXR5WPVLNEQ, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.591528813 (202h55m1.322796434s) [noise=true , address=79.127.201.215:4355 , piece_id=CIIRDUNHK2MHUQRRYD5NJOKE5X5ZTDSJQIBXGLX6PLMHAIRKYRBA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.591756432 (202h55m1.323024062s) [noise=true , address=77.90.31.13:4658 , piece_id=R6XVYXYB5QGEV4SYO2JMT3TOVX2CZKSWIWU6FMHLSI7XVM4T5DAA, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.592021882 (202h55m1.323289512s) [noise=true , address=77.90.31.11:4563 , piece_id=PSD3TD4YR2MAJBFXZFTEUFM45IE7EPLODIA3HRYEV3M3RAORZW5A, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.594283600 (202h55m1.32555123s) [noise=true , address=45.91.116.9:4272 , piece_id=MC63ZR3X76A334NQ2VQZAHGYIVW46YNX2V2RVBFEOHDN6I3WZ2MQ, seg_pos={0 0}, upload=211139]: Uploading piece... Aug 17 04:21:20.680272641 (202h55m1.411540291s) [upload=211139]: Next returned nil segment Aug 17 04:21:20.680287048 (202h55m1.411554698s) [upload=211139]: Waiting for error group managing segments... Aug 17 04:21:20.688508876 (202h55m1.419776496s) [noise=true , address=77.90.31.11:4563 , piece_id=PSD3TD4YR2MAJBFXZFTEUFM45IE7EPLODIA3HRYEV3M3RAORZW5A, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.689045385 (202h55m1.420313035s) [noise=true , address=45.91.116.6:4981 , piece_id=M3INZ2P4EMIHIN25MDOS6DZYOB2LLX4EEYP3LZXYWRXR5WPVLNEQ, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.689071676 (202h55m1.420339306s) [noise=true , address=45.91.116.9:4272 , piece_id=MC63ZR3X76A334NQ2VQZAHGYIVW46YNX2V2RVBFEOHDN6I3WZ2MQ, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.691218958 (202h55m1.422486608s) [noise=true , address=77.90.31.12:4621 , piece_id=JQ7M5YUHU5NVM6BF5EG2RJKQKMWYL2WIJ446DDXDJCRSMRXON27A, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.700201887 (202h55m1.431469517s) [noise=true , address=45.91.116.13:4184 , piece_id=EK5KUV3VCESUPXP7IPKYFN72X624GLPXKVJI3JNZCHNDFFPK3IXQ, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.720171059 (202h55m1.451438709s) [noise=true , address=77.90.31.13:4658 , piece_id=R6XVYXYB5QGEV4SYO2JMT3TOVX2CZKSWIWU6FMHLSI7XVM4T5DAA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.727782247 (202h55m1.459049867s) [noise=true , address=77.90.31.19:4872 , piece_id=CLAAH6J3YDZ3TJWF5YKKYM64IWJFV5ZZVY6QE4JO33G3DFF3REVA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:20.798259368 (202h55m1.529526998s) [noise=true , address=77.90.31.10:4523 , piece_id=4QU2XTD45JZLH6LYYHFQHRGYOF534T4VXU4LPEQKOLPJJ4B7ZMKA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:21.034027815 (202h55m1.765295465s) [noise=true , address=79.127.201.215:4355 , piece_id=CIIRDUNHK2MHUQRRYD5NJOKE5X5ZTDSJQIBXGLX6PLMHAIRKYRBA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:21:21.192573463 (202h55m1.923841093s) [noise=true , address=66.23.193.175:4553 , piece_id=4WTQ7KDTUUYJTYBAPDMMJTMS36BHXGWDUCID7ZFSIISJUNVIEPPQ, seg_pos={0 0}, upload=211139]: Done uploading piece. err: <nil> Aug 17 04:22:20.544583439 (202h56m1.275851089s) [noise=true , address=104.36.82.35:4612 , piece_id=4HEDST6C47ADCLUQ2B7IT3VVTDKZJ3ZR3TGXVX63SRZ34T5BTRXA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 5.161.89.252:35782->104.36.82.35:4612: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544621581 (202h56m1.275889231s) [noise=true , address=104.36.82.60:4360 , piece_id=76KZRVKEGTCRZKLI2IH6YUNGCNB64SNWBTBRHZCLZWWH7ZQVK46Q, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:12T24rtYBJ47E6Ef1ZZKMWcpBrWBpCg5QV2ZHniCZZkzJF2yUNk, address:104.36.82.60:4360): protocol: rpc: dial tcp 104.36.82.60:4360: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544659933 (202h56m1.275927583s) [noise=true , address=104.36.82.44:4883 , piece_id=ZDK7UCS365YHVABBUARN7TTEJTHW5YX42KR7D5E6DS53FDGOQ66Q, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:12cx1avL4umDm2pzJS1rNZqeDZjZMWnMtNsyNTdMdkxhpa1ByUN, address:104.36.82.44:4883): protocol: rpc: dial tcp 104.36.82.44:4883: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544696141 (202h56m1.275963801s) [noise=true , address=104.36.82.21:4508 , piece_id=RITMU5RREPWJKX35AG3E5C4QNZBPASKUMKPBOEP5N7WAL7UGBNUA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:1269NRnqpabU6H8q9vSCmE8yMzznD1L7dyZFmzCWhtzJaTjbXg3, address:104.36.82.21:4508): protocol: rpc: dial tcp 104.36.82.21:4508: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544727200 (202h56m1.27599485s) [noise=true , address=104.36.82.34:4557 , piece_id=AQPVQH7IKFYOVO3SY4JB3IQWEHRGTNQYQRZMFPHY4WZ7B336X7KA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT, address:104.36.82.34:4557): protocol: write tcp 5.161.89.252:26630->104.36.82.34:4557: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544807440 (202h56m1.276075091s) [noise=true , address=104.36.82.23:4474 , piece_id=NOKOTMAIA5JSXOL7AB6I5EOMNSRY7GTPNSXJAU3BIJNLTVNZPPHA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: rpc: dial tcp 104.36.82.23:4474: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544861072 (202h56m1.276128722s) [noise=true , address=104.36.82.16:4308 , piece_id=A6TQMADKLCOUU7HW36XCG4XMTPVG4DZXWOYN2HF5B2V6XXBSB5TQ, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:1PgA5SZkBvK673LsV7p2sXpbi1r79esNNZspPZTzaitqfbjtyY, address:104.36.82.16:4308): protocol: rpc: dial tcp 104.36.82.16:4308: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544871792 (202h56m1.276139443s) [noise=true , address=104.36.82.8:4992 , piece_id=UCOJ2KSSCBVEGVZGBVA7ISTPY4AFLP5XZYF42WM4K26VTS3WHWZA, seg_pos={0 0}, upload=211139]: Done uploading piece. err: ecclient: upload failed (node:1hbt8HKTunpPKyAzH4XBwGQqTKzcgqPpoWwKZm9JMtaK8P7aVp, address:104.36.82.8:4992): protocol: rpc: dial tcp 104.36.82.8:4992: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:20.544894385 (202h56m1.276162034s) [seg_pos={0 0}, upload=211139]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:22:20.558539311 (202h56m1.28980696s) [upload=211139]: Done uploading segments... Aug 17 04:22:20.558566281 (202h56m1.289833931s) [upload=211139]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.016690798s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.017435169s FAIL (2025-08-17 04:22:14 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:21:44.372109307 (202h55m25.103376957s) [upload=211145]: Starting upload Aug 17 04:21:44.372198896 (202h55m25.103466556s) [upload=211145]: Done starting upload Aug 17 04:21:44.372326306 (202h55m25.103593926s) [upload=211145]: Uploading segments... Aug 17 04:21:44.372590414 (202h55m25.103858063s) [upload=211145]: Got next segment. Inline: false Aug 17 04:21:44.372634096 (202h55m25.103901745s) [upload=211145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:44.372641550 (202h55m25.103909199s) [upload=211145]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:44.396833941 (202h55m25.128101571s) [seg_pos={0 0}, upload=211145]: Begin upload segment... Aug 17 04:21:44.396865430 (202h55m25.128133061s) [seg_pos={0 0}, upload=211145]: Done begin upload segment. Aug 17 04:21:44.396885127 (202h55m25.128152747s) [noise=true , address=104.36.82.8:4971 , piece_id=IUYIA7DM6NQSKLQTD44PJB5OHYWMA5ASOVBO3AM2JBZ5TBQLKHOA, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.396953906 (202h55m25.128221566s) [noise=true , address=104.36.82.10:4079 , piece_id=TNEYY5TD377DZD5J7EDHLRLBM6IFWMAY2H43J3JW62PW2U4NQ5UQ, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.397186735 (202h55m25.128454355s) [noise=true , address=45.91.116.9:4261 , piece_id=RTEYKWJAYZEQG5NMHNGWOK2B3X6XU4AVI3NET5LA7DRLQ3QFXW5A, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.397462965 (202h55m25.128730595s) [noise=true , address=104.36.82.14:4223 , piece_id=MOWXVCPOESFRY32UMW2QRFUFDCX2XZ5PKMGDDBSHVQZVPJVGBU2A, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.397635359 (202h55m25.128903009s) [noise=true , address=77.90.31.16:4733 , piece_id=TMA2KIFKHNK4ULZ26MDY7U6MZHEFYMX5LOD4C7GHXUJBMK7OROWQ, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.397686165 (202h55m25.128953785s) [noise=true , address=45.91.116.5:4299 , piece_id=AP4AZ2UEFAYYYUQVQIULMHBDIZ5IENFQI7DHRQZE7EVDACKZ3F4A, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.397922319 (202h55m25.12918994s) [noise=true , address=104.36.82.9:4007 , piece_id=HQGMPAIK4RO52HFFNWBO5LF4262ZIDRVNUCMDF43DL7JY5Z22GMQ, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.398129399 (202h55m25.12939702s) [noise=true , address=77.90.31.15:4697 , piece_id=SUZ4SGHK26AYEAQWK7YLULE4FRHIFZ6CBUZ2JTIWXNMRBH4M4NKA, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.398147524 (202h55m25.129415174s) [noise=true , address=77.90.31.18:4806 , piece_id=DY7XBBWABIFIVUJTCZKOCIG5ZJPPMP7XMSXGVFGA6FIAK7XOSJDA, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.398391081 (202h55m25.129658712s) [noise=true , address=45.91.116.14:4191 , piece_id=RRM4VWAOHCMTXMKEZYROQHINJ3QB734VDWEYCTJWMV2PNHADZM4A, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.398633739 (202h55m25.129901389s) [noise=true , address=104.36.82.16:4295 , piece_id=GQ4DEJE5GY6GFBSTC4C4QPTUHQ2H2PBGYBMVDOM33RUGMOQNKCYA, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.398646332 (202h55m25.129913962s) [noise=true , address=104.36.82.10:4077 , piece_id=DP6OUWPWXW7AGL3Q546ABF2CLU7NUEWQD7DF2IZMP2Y4CP4WKZ2A, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.399202569 (202h55m25.13047022s) [noise=true , address=45.91.116.6:4973 , piece_id=25NNZVWYKXQ5I6UQXB6AN65LYPP3PZSDW5MCKX25GLM53V2KMVUQ, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.399659069 (202h55m25.130926729s) [noise=true , address=45.91.116.4:4938 , piece_id=F43POI5OIL3VODPOGVNO2A4YKS6YC4IF4Y2DL4NVYFLR4CFVF2CA, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.400299845 (202h55m25.131567505s) [noise=true , address=77.90.31.13:4628 , piece_id=3KGJJDWL7QZAJLZSBYOJ3DJIA2VG3Q52M65W3WQIYMZDIK5ZTUCA, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.401730739 (202h55m25.132998389s) [noise=true , address=45.91.116.6:4974 , piece_id=BVQBG7FKSTU3F2DBRQAQDPBHC2GRORXA5FSOWE7LTMTUKFSUOINQ, seg_pos={0 0}, upload=211145]: Uploading piece... Aug 17 04:21:44.505162045 (202h55m25.236429695s) [upload=211145]: Next returned nil segment Aug 17 04:21:44.505178647 (202h55m25.236446296s) [upload=211145]: Waiting for error group managing segments... Aug 17 04:21:44.515725801 (202h55m25.246993462s) [noise=true , address=45.91.116.4:4938 , piece_id=F43POI5OIL3VODPOGVNO2A4YKS6YC4IF4Y2DL4NVYFLR4CFVF2CA, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.516391424 (202h55m25.247659084s) [noise=true , address=45.91.116.6:4974 , piece_id=BVQBG7FKSTU3F2DBRQAQDPBHC2GRORXA5FSOWE7LTMTUKFSUOINQ, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.516616498 (202h55m25.247884138s) [noise=true , address=45.91.116.9:4261 , piece_id=RTEYKWJAYZEQG5NMHNGWOK2B3X6XU4AVI3NET5LA7DRLQ3QFXW5A, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.517066836 (202h55m25.248334485s) [noise=true , address=45.91.116.14:4191 , piece_id=RRM4VWAOHCMTXMKEZYROQHINJ3QB734VDWEYCTJWMV2PNHADZM4A, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.517877031 (202h55m25.249144681s) [noise=true , address=77.90.31.18:4806 , piece_id=DY7XBBWABIFIVUJTCZKOCIG5ZJPPMP7XMSXGVFGA6FIAK7XOSJDA, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.519365983 (202h55m25.250633634s) [noise=true , address=45.91.116.6:4973 , piece_id=25NNZVWYKXQ5I6UQXB6AN65LYPP3PZSDW5MCKX25GLM53V2KMVUQ, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.519623038 (202h55m25.250890688s) [noise=true , address=77.90.31.15:4697 , piece_id=SUZ4SGHK26AYEAQWK7YLULE4FRHIFZ6CBUZ2JTIWXNMRBH4M4NKA, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.519996400 (202h55m25.25126405s) [noise=true , address=45.91.116.5:4299 , piece_id=AP4AZ2UEFAYYYUQVQIULMHBDIZ5IENFQI7DHRQZE7EVDACKZ3F4A, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.521170421 (202h55m25.252438071s) [noise=true , address=77.90.31.16:4733 , piece_id=TMA2KIFKHNK4ULZ26MDY7U6MZHEFYMX5LOD4C7GHXUJBMK7OROWQ, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:21:44.641084154 (202h55m25.372351805s) [noise=true , address=77.90.31.13:4628 , piece_id=3KGJJDWL7QZAJLZSBYOJ3DJIA2VG3Q52M65W3WQIYMZDIK5ZTUCA, seg_pos={0 0}, upload=211145]: Done uploading piece. err: <nil> Aug 17 04:22:14.354761198 (202h55m55.086028818s) [noise=true , address=104.36.82.10:4077 , piece_id=DP6OUWPWXW7AGL3Q546ABF2CLU7NUEWQD7DF2IZMP2Y4CP4WKZ2A, seg_pos={0 0}, upload=211145]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: rpc: dial tcp 104.36.82.10:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:14.354815751 (202h55m55.086083371s) [noise=true , address=104.36.82.16:4295 , piece_id=GQ4DEJE5GY6GFBSTC4C4QPTUHQ2H2PBGYBMVDOM33RUGMOQNKCYA, seg_pos={0 0}, upload=211145]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:104.36.82.16:4295): protocol: rpc: dial tcp 104.36.82.16:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:14.354824588 (202h55m55.086092207s) [noise=true , address=104.36.82.10:4079 , piece_id=TNEYY5TD377DZD5J7EDHLRLBM6IFWMAY2H43J3JW62PW2U4NQ5UQ, seg_pos={0 0}, upload=211145]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:14.354853402 (202h55m55.086121032s) [noise=true , address=104.36.82.9:4007 , piece_id=HQGMPAIK4RO52HFFNWBO5LF4262ZIDRVNUCMDF43DL7JY5Z22GMQ, seg_pos={0 0}, upload=211145]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:14.354859723 (202h55m55.086127353s) [noise=true , address=104.36.82.14:4223 , piece_id=MOWXVCPOESFRY32UMW2QRFUFDCX2XZ5PKMGDDBSHVQZVPJVGBU2A, seg_pos={0 0}, upload=211145]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 5.161.89.252:18598->104.36.82.14:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:14.354864022 (202h55m55.086131652s) [noise=true , address=104.36.82.8:4971 , piece_id=IUYIA7DM6NQSKLQTD44PJB5OHYWMA5ASOVBO3AM2JBZ5TBQLKHOA, seg_pos={0 0}, upload=211145]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 5.161.89.252:47836->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:22:14.354889369 (202h55m55.086156989s) [seg_pos={0 0}, upload=211145]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:22:14.370958137 (202h55m55.102225767s) [upload=211145]: Done uploading segments... Aug 17 04:22:14.370967415 (202h55m55.102235045s) [upload=211145]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.017455898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.019718336s FAIL (2025-08-17 04:21:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 04:20:58.622986735 (202h54m39.354254385s) [upload=211130]: Starting upload Aug 17 04:20:58.623045626 (202h54m39.354313277s) [upload=211130]: Done starting upload Aug 17 04:20:58.623106220 (202h54m39.35437387s) [upload=211130]: Uploading segments... Aug 17 04:20:58.623378353 (202h54m39.354646002s) [upload=211130]: Got next segment. Inline: false Aug 17 04:20:58.623419219 (202h54m39.354686869s) [upload=211130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:20:58.623426563 (202h54m39.354694243s) [upload=211130]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:20:58.654875335 (202h54m39.386142985s) [seg_pos={0 0}, upload=211130]: Begin upload segment... Aug 17 04:20:58.654923966 (202h54m39.386191616s) [seg_pos={0 0}, upload=211130]: Done begin upload segment. Aug 17 04:20:58.654955035 (202h54m39.386222684s) [noise=true , address=104.36.82.17:4333 , piece_id=QOISEHJ7DT46XTZPW6MTAVE4P2NDBKTDSKIDD3S53LYNOWK23S6Q, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.654988137 (202h54m39.386255786s) [noise=true , address=104.36.82.5:4863 , piece_id=Q6RT6OZLA7OYJOD4XK2XJBAWDOAINR33ZBB3X7LZOB2XZUFHZTJA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.655399401 (202h54m39.386667051s) [noise=true , address=104.36.82.39:4758 , piece_id=ELI3RBJLJLY3CCPC265CJFLO5QKWOIGEEHMWSF3EGJZ3BWHAH3SA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.655409560 (202h54m39.38667721s) [noise=true , address=45.91.116.13:4184 , piece_id=F2SKBNWPUWY7VJNSCMS4LMT4IH4QMUMGDUN5PQWFWQXMIJRV2WQQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.655740333 (202h54m39.387007984s) [noise=true , address=104.36.82.19:4403 , piece_id=X2T6RYA7XUHMM67NM24C3KNRHO36W3KRQ35V7777I6XHGGVL7SSA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.655844779 (202h54m39.387112429s) [noise=true , address=66.23.193.175:4557 , piece_id=OKRV6AJALTXBVP7S4VFP273R6B7GE2TRJMM2CEP5DLKYCOYJ4ZXA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.656195009 (202h54m39.387462659s) [noise=true , address=79.127.201.215:4356 , piece_id=FZDWQIXNRHIDWZHEMPLTXQZULKTAOVJRNG3G6X2D54UIC5ZFJ55Q, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.656220337 (202h54m39.387487977s) [noise=true , address=104.36.82.4:4807 , piece_id=SF5732MIVFQDV522P3ERPRD5CLHFD4E3VLRPOYH44SUFOLKFEOAQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.656589993 (202h54m39.387857643s) [noise=true , address=79.127.205.235:4675 , piece_id=JTLHSZPMX4LGWX2R7MSLBJYEHR2SBV5LMWE7SYLAWFMK7IPK4NBQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.656595814 (202h54m39.387863463s) [noise=true , address=77.90.31.16:4741 , piece_id=B56TOKB56GWJ5SAEULWJJPMSLG4UMOKMT7ZOLHSNHFNWIXFNRQDA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.656955951 (202h54m39.388223601s) [noise=true , address=104.36.82.22:4514 , piece_id=FOV5LG6PVQUP2GPHSN3LOQERA7YMD7QOLB2MKA55F5KQ7OMRLUYA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.657017096 (202h54m39.388284746s) [noise=true , address=77.90.31.15:4708 , piece_id=VP4HOFOF7A3SAS2YZBA5QKIIUV66RFRDQW4ACKJS535HLZHGMTQQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.657289689 (202h54m39.388557329s) [noise=true , address=45.91.116.6:4976 , piece_id=SZY6S74MX577CJWRXZD2KH2DIZ2KTGZJZUOVFUPJI2WLFNRUZOXQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.657686176 (202h54m39.388953826s) [noise=true , address=104.36.82.60:4370 , piece_id=DOAETJRY7B62OXSHAIM2LUAIZR4EXD7VGUPNFQREPT4XUVQ3ZLEQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.658528311 (202h54m39.389795961s) [noise=true , address=77.90.31.13:4629 , piece_id=VASSL7AI5JCVYUGNZLM3IULV3PRGJAKPQBARWD64ASEQMGZ6AWLA, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.661599081 (202h54m39.392866741s) [noise=true , address=104.36.82.23:4455 , piece_id=U5KOPRAUGTMNOTTNQ5WAFZG5WIAREGVXZDGB457KIQVEJCS5YDNQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.662377568 (202h54m39.393645227s) [noise=true , address=45.91.116.7:4023 , piece_id=HBIAK5NULYXTXUMTZN7PHW5JERFWBZO6JSJ6OZCDHD2RFJO3Z47A, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.662801816 (202h54m39.394069446s) [noise=true , address=79.127.220.101:4068 , piece_id=X6JOBJ4WYLAQICPTYMUC2NXGZTUBDBXANPQOEFNH2HQIKYKG37NQ, seg_pos={0 0}, upload=211130]: Uploading piece... Aug 17 04:20:58.721049475 (202h54m39.452317105s) [upload=211130]: Next returned nil segment Aug 17 04:20:58.721085553 (202h54m39.452353173s) [upload=211130]: Waiting for error group managing segments... Aug 17 04:20:58.731124571 (202h54m39.462392201s) [noise=true , address=45.91.116.6:4976 , piece_id=SZY6S74MX577CJWRXZD2KH2DIZ2KTGZJZUOVFUPJI2WLFNRUZOXQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:58.732887529 (202h54m39.464155159s) [noise=true , address=45.91.116.7:4023 , piece_id=HBIAK5NULYXTXUMTZN7PHW5JERFWBZO6JSJ6OZCDHD2RFJO3Z47A, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:58.750348210 (202h54m39.48161583s) [noise=true , address=77.90.31.16:4741 , piece_id=B56TOKB56GWJ5SAEULWJJPMSLG4UMOKMT7ZOLHSNHFNWIXFNRQDA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:58.774588812 (202h54m39.505856433s) [noise=true , address=77.90.31.15:4708 , piece_id=VP4HOFOF7A3SAS2YZBA5QKIIUV66RFRDQW4ACKJS535HLZHGMTQQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:58.806038024 (202h54m39.537305654s) [noise=true , address=45.91.116.13:4184 , piece_id=F2SKBNWPUWY7VJNSCMS4LMT4IH4QMUMGDUN5PQWFWQXMIJRV2WQQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:58.821513506 (202h54m39.552781136s) [noise=true , address=77.90.31.13:4629 , piece_id=VASSL7AI5JCVYUGNZLM3IULV3PRGJAKPQBARWD64ASEQMGZ6AWLA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:58.835492631 (202h54m39.566760251s) [noise=true , address=79.127.220.101:4068 , piece_id=X6JOBJ4WYLAQICPTYMUC2NXGZTUBDBXANPQOEFNH2HQIKYKG37NQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:59.045906350 (202h54m39.777173969s) [noise=true , address=79.127.201.215:4356 , piece_id=FZDWQIXNRHIDWZHEMPLTXQZULKTAOVJRNG3G6X2D54UIC5ZFJ55Q, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:59.179572913 (202h54m39.910840552s) [noise=true , address=79.127.205.235:4675 , piece_id=JTLHSZPMX4LGWX2R7MSLBJYEHR2SBV5LMWE7SYLAWFMK7IPK4NBQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:20:59.253915211 (202h54m39.985182841s) [noise=true , address=66.23.193.175:4557 , piece_id=OKRV6AJALTXBVP7S4VFP273R6B7GE2TRJMM2CEP5DLKYCOYJ4ZXA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: <nil> Aug 17 04:21:58.603082534 (202h55m39.334350195s) [noise=true , address=104.36.82.23:4455 , piece_id=U5KOPRAUGTMNOTTNQ5WAFZG5WIAREGVXZDGB457KIQVEJCS5YDNQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm, address:104.36.82.23:4455): protocol: rpc: dial tcp 104.36.82.23:4455: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603152346 (202h55m39.334419997s) [noise=true , address=104.36.82.5:4863 , piece_id=Q6RT6OZLA7OYJOD4XK2XJBAWDOAINR33ZBB3X7LZOB2XZUFHZTJA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn, address:104.36.82.5:4863): protocol: write tcp 5.161.89.252:17326->104.36.82.5:4863: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603288483 (202h55m39.334556133s) [noise=true , address=104.36.82.39:4758 , piece_id=ELI3RBJLJLY3CCPC265CJFLO5QKWOIGEEHMWSF3EGJZ3BWHAH3SA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:12XjsLCQGSuQkDmyC4QFENshoLeo4kCZG1YNm34wBvG8UEuxTG7, address:104.36.82.39:4758): protocol: rpc: dial tcp 104.36.82.39:4758: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603351681 (202h55m39.334619332s) [noise=true , address=104.36.82.22:4514 , piece_id=FOV5LG6PVQUP2GPHSN3LOQERA7YMD7QOLB2MKA55F5KQ7OMRLUYA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:104.36.82.22:4514): protocol: rpc: dial tcp 104.36.82.22:4514: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603403008 (202h55m39.334670658s) [noise=true , address=104.36.82.17:4333 , piece_id=QOISEHJ7DT46XTZPW6MTAVE4P2NDBKTDSKIDD3S53LYNOWK23S6Q, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:1xRdTZoEk3NsnS445GcdEvwAdDzNLwUYSfgjB7bBSGRx9L9mrU, address:104.36.82.17:4333): protocol: rpc: dial tcp 104.36.82.17:4333: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603428246 (202h55m39.334695896s) [noise=true , address=104.36.82.60:4370 , piece_id=DOAETJRY7B62OXSHAIM2LUAIZR4EXD7VGUPNFQREPT4XUVQ3ZLEQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:14LBgixGGgNeYs63ZSpkYJmrEGr3W9Ma1s2Sj8cCxU6mUxm1aC, address:104.36.82.60:4370): protocol: rpc: dial tcp 104.36.82.60:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603440008 (202h55m39.334707658s) [noise=true , address=104.36.82.19:4403 , piece_id=X2T6RYA7XUHMM67NM24C3KNRHO36W3KRQ35V7777I6XHGGVL7SSA, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:12SWX9WeN8GXMsV3tUyArHw4As5vKwJRzyMxPZRMug4iAY42vh4, address:104.36.82.19:4403): protocol: rpc: dial tcp 104.36.82.19:4403: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603483229 (202h55m39.334750879s) [noise=true , address=104.36.82.4:4807 , piece_id=SF5732MIVFQDV522P3ERPRD5CLHFD4E3VLRPOYH44SUFOLKFEOAQ, seg_pos={0 0}, upload=211130]: Done uploading piece. err: ecclient: upload failed (node:1MZ9LJLHF4XXvmRG2r7XkanDfdNSKCUgua9EBPusPuGyLzTLo8, address:104.36.82.4:4807): protocol: write tcp 5.161.89.252:55798->104.36.82.4:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:58.603537962 (202h55m39.334805612s) [seg_pos={0 0}, upload=211130]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:21:58.620788305 (202h55m39.352055955s) [upload=211130]: Done uploading segments... Aug 17 04:21:58.620835403 (202h55m39.352103053s) [upload=211130]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.019733215s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.018463627s FAIL (2025-08-17 04:21:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Aug 17 04:21:09.629171232 (202h54m50.360438862s) [upload=211134]: Starting upload Aug 17 04:21:09.629240713 (202h54m50.360508343s) [upload=211134]: Done starting upload Aug 17 04:21:09.629311868 (202h54m50.360579498s) [upload=211134]: Uploading segments... Aug 17 04:21:09.629468372 (202h54m50.360736002s) [upload=211134]: Got next segment. Inline: false Aug 17 04:21:09.629515541 (202h54m50.360783171s) [upload=211134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:21:09.629520169 (202h54m50.360787799s) [upload=211134]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:21:09.655201123 (202h54m50.386468753s) [seg_pos={0 0}, upload=211134]: Begin upload segment... Aug 17 04:21:09.655237120 (202h54m50.386504741s) [seg_pos={0 0}, upload=211134]: Done begin upload segment. Aug 17 04:21:09.655257339 (202h54m50.386524959s) [noise=true , address=104.36.82.7:4912 , piece_id=4HQ3KSP6ZAYG3X4TSOFTPTDVKYXV4ACFJVRUI4R5PMAQHWMWMF7A, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.655290000 (202h54m50.38655763s) [noise=true , address=104.36.82.4:4802 , piece_id=VUFYPSAG274AYMI7LJG3KWJCQSISA5MBGRAELSZSSLKNXBL5RSDA, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.655540943 (202h54m50.386808573s) [noise=true , address=45.91.116.10:4081 , piece_id=M356JMWXW6HA4QHNIPRGZNGC57W64PJCNXWI5UNPQ2XKEUNXRXHA, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.655570709 (202h54m50.386838479s) [noise=true , address=77.90.31.14:4661 , piece_id=Z47ZOIP4NEUA6EHNNUWU4OIZEIP5AKYW5VVA2FE5X2KPCVK3GSHA, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.655764383 (202h54m50.387032013s) [noise=true , address=77.90.31.10:4520 , piece_id=QPNKTM6F4AIT3DLZPI74WHP3QRUDS2SVLYVSBTDZL334IPV2RUKA, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.655794670 (202h54m50.38706229s) [noise=true , address=45.91.116.4:4940 , piece_id=Q3M4JMWNNMGQFZ2D7XWZDIF3NAOEWX7RUGSNDDDWBI5OOPRMQ7IQ, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656015096 (202h54m50.387282715s) [noise=true , address=104.36.82.2:4729 , piece_id=WX4LCZHGV43J6NI37T47J6BJWAVBAJEQYUZUQIDI4XX4FI7PBGCQ, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656077613 (202h54m50.387345243s) [noise=true , address=45.91.116.5:4297 , piece_id=OACYXSVZMH6F7FX2ZEA57273BVOUK3FEOC4KQVMQBW7CA2DRDVOQ, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656226222 (202h54m50.387493852s) [noise=true , address=77.90.31.11:4555 , piece_id=6NP4KIWSCC3HCKAZZTVTYB4QFR6D7YJXQUSFFDKHNISOBVEKQG4A, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656292567 (202h54m50.387560197s) [noise=true , address=45.91.116.13:4153 , piece_id=7QHFLUSE4BKOOKRISXGKU5YGC2YK37DO2C6LXAR44FUMSITNX73A, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656430818 (202h54m50.387698448s) [noise=true , address=45.91.116.7:4011 , piece_id=QXX7YPFQ4FL7IZLT5I6RHZY66H3VDBIBOKT5O7BBO75CS7TKGUFQ, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656496411 (202h54m50.387764041s) [noise=true , address=45.91.116.8:4048 , piece_id=TA4DEEPS2QN4SA4GTNKFO7E7TSA643NBJ7NGB5LIENFKTQRMXS7Q, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656673975 (202h54m50.387941605s) [noise=true , address=45.91.116.6:4974 , piece_id=76LYFMBUPYMQGWZHTMNRXGUHJSNCQ6EONBIXRA67ZRYEVVGDLVXQ, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656708741 (202h54m50.387976361s) [noise=true , address=104.36.82.8:4971 , piece_id=5EIDYCHAAYCBGLOLQGBLNGO66PPDEJFEKAZMOHUGZG5ECXMHHZYA, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.656906794 (202h54m50.388174424s) [noise=true , address=77.90.31.18:4807 , piece_id=W2H5FFXM2A4ICIOWTHTMU2YXUPCE4AHD24PGL5WH5R7XYKQEM7IQ, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.657122761 (202h54m50.388390381s) [noise=true , address=104.36.82.11:4115 , piece_id=UUDC6YHU3EBTCABSXOHHNHJD2UMQMOMKY7LTCS3LGUQ4PWVLEDNA, seg_pos={0 0}, upload=211134]: Uploading piece... Aug 17 04:21:09.735482571 (202h54m50.46675019s) [upload=211134]: Next returned nil segment Aug 17 04:21:09.735493741 (202h54m50.466761371s) [upload=211134]: Waiting for error group managing segments... Aug 17 04:21:09.744486450 (202h54m50.47575408s) [noise=true , address=77.90.31.18:4807 , piece_id=W2H5FFXM2A4ICIOWTHTMU2YXUPCE4AHD24PGL5WH5R7XYKQEM7IQ, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.745307095 (202h54m50.476574715s) [noise=true , address=45.91.116.8:4048 , piece_id=TA4DEEPS2QN4SA4GTNKFO7E7TSA643NBJ7NGB5LIENFKTQRMXS7Q, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.745713500 (202h54m50.47698113s) [noise=true , address=45.91.116.7:4011 , piece_id=QXX7YPFQ4FL7IZLT5I6RHZY66H3VDBIBOKT5O7BBO75CS7TKGUFQ, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.746120586 (202h54m50.477388216s) [noise=true , address=45.91.116.5:4297 , piece_id=OACYXSVZMH6F7FX2ZEA57273BVOUK3FEOC4KQVMQBW7CA2DRDVOQ, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.746492897 (202h54m50.477760527s) [noise=true , address=45.91.116.4:4940 , piece_id=Q3M4JMWNNMGQFZ2D7XWZDIF3NAOEWX7RUGSNDDDWBI5OOPRMQ7IQ, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.746891758 (202h54m50.478159388s) [noise=true , address=45.91.116.6:4974 , piece_id=76LYFMBUPYMQGWZHTMNRXGUHJSNCQ6EONBIXRA67ZRYEVVGDLVXQ, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.747307100 (202h54m50.47857472s) [noise=true , address=77.90.31.10:4520 , piece_id=QPNKTM6F4AIT3DLZPI74WHP3QRUDS2SVLYVSBTDZL334IPV2RUKA, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.747676385 (202h54m50.478944015s) [noise=true , address=45.91.116.10:4081 , piece_id=M356JMWXW6HA4QHNIPRGZNGC57W64PJCNXWI5UNPQ2XKEUNXRXHA, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.748087780 (202h54m50.479355399s) [noise=true , address=77.90.31.11:4555 , piece_id=6NP4KIWSCC3HCKAZZTVTYB4QFR6D7YJXQUSFFDKHNISOBVEKQG4A, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.748461944 (202h54m50.479729574s) [noise=true , address=45.91.116.13:4153 , piece_id=7QHFLUSE4BKOOKRISXGKU5YGC2YK37DO2C6LXAR44FUMSITNX73A, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:09.749311734 (202h54m50.480579364s) [noise=true , address=77.90.31.14:4661 , piece_id=Z47ZOIP4NEUA6EHNNUWU4OIZEIP5AKYW5VVA2FE5X2KPCVK3GSHA, seg_pos={0 0}, upload=211134]: Done uploading piece. err: <nil> Aug 17 04:21:39.612044751 (202h55m20.343312381s) [noise=true , address=104.36.82.11:4115 , piece_id=UUDC6YHU3EBTCABSXOHHNHJD2UMQMOMKY7LTCS3LGUQ4PWVLEDNA, seg_pos={0 0}, upload=211134]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:39.612102190 (202h55m20.34336981s) [noise=true , address=104.36.82.4:4802 , piece_id=VUFYPSAG274AYMI7LJG3KWJCQSISA5MBGRAELSZSSLKNXBL5RSDA, seg_pos={0 0}, upload=211134]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 5.161.89.252:18284->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:39.612122859 (202h55m20.343390489s) [noise=true , address=104.36.82.2:4729 , piece_id=WX4LCZHGV43J6NI37T47J6BJWAVBAJEQYUZUQIDI4XX4FI7PBGCQ, seg_pos={0 0}, upload=211134]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: write tcp 5.161.89.252:26108->104.36.82.2:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:39.612145541 (202h55m20.343413161s) [noise=true , address=104.36.82.7:4912 , piece_id=4HQ3KSP6ZAYG3X4TSOFTPTDVKYXV4ACFJVRUI4R5PMAQHWMWMF7A, seg_pos={0 0}, upload=211134]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: rpc: dial tcp 104.36.82.7:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:39.612159407 (202h55m20.343427037s) [noise=true , address=104.36.82.8:4971 , piece_id=5EIDYCHAAYCBGLOLQGBLNGO66PPDEJFEKAZMOHUGZG5ECXMHHZYA, seg_pos={0 0}, upload=211134]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: rpc: dial tcp 104.36.82.8:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:39.612178613 (202h55m20.343446233s) [seg_pos={0 0}, upload=211134]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Aug 17 04:21:39.629442593 (202h55m20.360710213s) [upload=211134]: Done uploading segments... Aug 17 04:21:39.629455818 (202h55m20.360723448s) [upload=211134]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.018480479s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 571.379315ms FAIL (2025-08-17 04:21:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 571.401077ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.025772913s FAIL (2025-08-17 04:21:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 8) ======LOGS====== PHWUJKPP6Q2A, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.548778010 (202h54m1.28004566s) [noise=true , address=104.36.82.35:4591 , piece_id=W2D3YMIKVJ5R67NTF7FN5SECHSP6D5FFAWS76WMNCX5AMGKCF2CQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.549140473 (202h54m1.280408123s) [noise=true , address=104.36.82.5:4857 , piece_id=ZXKQEZHM4OEOQDOSUZN7S5BJTZRNP7INF2LCXHMTWXSUFQ7K6V7Q, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.549180718 (202h54m1.280448379s) [noise=true , address=104.36.82.58:4293 , piece_id=4KJQRM6BEFJGBG554JZWGM2KMGWBH5SD2AWFMM5SEEPPIHHLTODA, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.549536257 (202h54m1.280803907s) [noise=true , address=45.91.116.6:4976 , piece_id=Z7KK45J7RG5P567CQHZKQXRKWADVXVC3UYGHUYJHG2FOHIR7LOWQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.549572236 (202h54m1.280839895s) [noise=true , address=104.36.82.44:4878 , piece_id=AOOX6PMAUZLJC77FQBEJBVHOEQCN3VMIA7H5V4RXNGNDP3I3NO6Q, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.549949695 (202h54m1.281217346s) [noise=true , address=77.90.31.10:4539 , piece_id=NKGORP6X5JSJTTC4BAK7CMLCXX6VZQC62J64RXAEIOH6YBQ6C47A, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.549965906 (202h54m1.281233556s) [noise=true , address=104.36.82.23:4446 , piece_id=QT6SIV7BIEJ5ZSTZZMQEG7D2O2XIQ2M5QGLUZKMDTTJWO67MCECQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.550325864 (202h54m1.281593514s) [noise=true , address=77.90.31.18:4828 , piece_id=FKJEIL534QH6IRBKRROPBZBHJW5ONGHVOI63NFCFJFNKDZ2U6OLQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.550374214 (202h54m1.281641865s) [noise=true , address=79.127.201.215:4359 , piece_id=C3YWXIIZONZ55H6SBDKC3PHQPDSYMVI4HS6RNLDXTJJDKDJH2NRA, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.550675712 (202h54m1.281943362s) [noise=true , address=104.36.82.60:4370 , piece_id=C76B5FVJPQ3V7LQ4ZWHBYV7HYI3HCVFPBLFZJNX5LIKXPPDVI4VQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.550758578 (202h54m1.282026229s) [noise=true , address=77.90.31.12:4604 , piece_id=OIHZWACP3NSSIDPH6WXIA3Q3E3AYBTVX32MT6E3EXNEOJPXSFHOQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.551049286 (202h54m1.282316936s) [noise=true , address=104.36.82.38:4718 , piece_id=AAB46JNYRBYHO5ATHGN23CTJKC4IMMWVHJKDSWUSMHVVHF6KZBKQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.551215629 (202h54m1.282483289s) [noise=true , address=79.127.201.215:4360 , piece_id=V7EHUFGIRBW2JP5MWN5BAKG4KPH4N3PHXSVHV7XSYDAPUXEXAJKQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.551394866 (202h54m1.282662516s) [noise=true , address=66.23.193.175:4563 , piece_id=KQXH4IUPSF6CURUWYNWFPWBUJOZAU4Q7SPPFCME7HJAO4M6MY6NQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.551631993 (202h54m1.282899643s) [noise=true , address=104.36.82.59:4301 , piece_id=43BEE22PQJ6FASHQPQCSVWH6DYGWUSL33AZ6PSNVNHVO2WXJDPOQ, seg_pos={0 0}, upload=211119]: Uploading piece... Aug 17 04:20:20.651262665 (202h54m1.382530315s) [upload=211119]: Next returned nil segment Aug 17 04:20:20.651277292 (202h54m1.382544952s) [upload=211119]: Waiting for error group managing segments... Aug 17 04:20:20.660157479 (202h54m1.391425129s) [noise=true , address=45.91.116.6:4976 , piece_id=Z7KK45J7RG5P567CQHZKQXRKWADVXVC3UYGHUYJHG2FOHIR7LOWQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:20.747735772 (202h54m1.479003422s) [noise=true , address=45.91.116.10:4102 , piece_id=7POXM7PSFA6ESEKGXUR3FNL3IRKZLA5NV24RRU3APHWUJKPP6Q2A, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:20.757279368 (202h54m1.488546988s) [noise=true , address=77.90.31.18:4828 , piece_id=FKJEIL534QH6IRBKRROPBZBHJW5ONGHVOI63NFCFJFNKDZ2U6OLQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:20.934146585 (202h54m1.665414204s) [noise=true , address=79.127.201.215:4360 , piece_id=V7EHUFGIRBW2JP5MWN5BAKG4KPH4N3PHXSVHV7XSYDAPUXEXAJKQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:21.015207310 (202h54m1.74647497s) [noise=true , address=77.90.31.10:4539 , piece_id=NKGORP6X5JSJTTC4BAK7CMLCXX6VZQC62J64RXAEIOH6YBQ6C47A, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:21.016318391 (202h54m1.747586041s) [noise=true , address=77.90.31.12:4604 , piece_id=OIHZWACP3NSSIDPH6WXIA3Q3E3AYBTVX32MT6E3EXNEOJPXSFHOQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:21.087389730 (202h54m1.818657381s) [noise=true , address=79.127.201.215:4359 , piece_id=C3YWXIIZONZ55H6SBDKC3PHQPDSYMVI4HS6RNLDXTJJDKDJH2NRA, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:20:21.149037060 (202h54m1.88030468s) [noise=true , address=66.23.193.175:4563 , piece_id=KQXH4IUPSF6CURUWYNWFPWBUJOZAU4Q7SPPFCME7HJAO4M6MY6NQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: <nil> Aug 17 04:21:20.456493896 (202h55m1.187761526s) [noise=true , address=104.36.82.59:4301 , piece_id=43BEE22PQJ6FASHQPQCSVWH6DYGWUSL33AZ6PSNVNHVO2WXJDPOQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:18HkaSrdHHnvmeNz1ormfcSBZH8bMUgkX7X3JmGuqkmtBoak29, address:104.36.82.59:4301): protocol: rpc: dial tcp 104.36.82.59:4301: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456507451 (202h55m1.187775071s) [noise=true , address=104.36.82.58:4293 , piece_id=4KJQRM6BEFJGBG554JZWGM2KMGWBH5SD2AWFMM5SEEPPIHHLTODA, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:15i783VxxzbriD7LZCbFWNBY8uCKUffXtmWE2eu2VupgE1t9bZ, address:104.36.82.58:4293): protocol: rpc: dial tcp 104.36.82.58:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456537477 (202h55m1.187805107s) [noise=true , address=104.36.82.9:4025 , piece_id=T2DKGKA46ZXOL4RUEMILFNIIE7VISABXG2VOGO4D5W4UMUHY2HVQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: rpc: dial tcp 104.36.82.9:4025: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456556232 (202h55m1.187823862s) [noise=true , address=104.36.82.35:4591 , piece_id=W2D3YMIKVJ5R67NTF7FN5SECHSP6D5FFAWS76WMNCX5AMGKCF2CQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:12a453d6xSb26epbgqPuco3UPTf9yzoxBeKToCUMNiNHUddAq55, address:104.36.82.35:4591): protocol: rpc: dial tcp 104.36.82.35:4591: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456569277 (202h55m1.187836907s) [noise=true , address=104.36.82.23:4446 , piece_id=QT6SIV7BIEJ5ZSTZZMQEG7D2O2XIQ2M5QGLUZKMDTTJWO67MCECQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:104.36.82.23:4446): protocol: rpc: dial tcp 104.36.82.23:4446: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456588363 (202h55m1.187855983s) [noise=true , address=104.36.82.44:4878 , piece_id=AOOX6PMAUZLJC77FQBEJBVHOEQCN3VMIA7H5V4RXNGNDP3I3NO6Q, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:1wX9xRmyv3KWVyJhFmgBt9uYqDU6U768dqHagKZeKBSEnFup5y, address:104.36.82.44:4878): protocol: rpc: dial tcp 104.36.82.44:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456598181 (202h55m1.187865811s) [noise=true , address=104.36.82.36:4634 , piece_id=MM3YJT6NVZAZSVPXEG454NX4OF44TZAYWG7GJEHOAIUXHRBCPR3A, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:1kwydp3q2wTrCVadJZB7WmKGK8h2R5QL7HGpUsjTpRbcnWUSb5, address:104.36.82.36:4634): protocol: rpc: dial tcp 104.36.82.36:4634: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456622558 (202h55m1.187890188s) [noise=true , address=104.36.82.38:4718 , piece_id=AAB46JNYRBYHO5ATHGN23CTJKC4IMMWVHJKDSWUSMHVVHF6KZBKQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:12dwmwypGjZv8TinKXYSfCEU4z7eSRML4GsXCBypzSRq6WnBs98, address:104.36.82.38:4718): protocol: rpc: dial tcp 104.36.82.38:4718: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456627717 (202h55m1.187895338s) [noise=true , address=104.36.82.60:4370 , piece_id=C76B5FVJPQ3V7LQ4ZWHBYV7HYI3HCVFPBLFZJNX5LIKXPPDVI4VQ, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:14LBgixGGgNeYs63ZSpkYJmrEGr3W9Ma1s2Sj8cCxU6mUxm1aC, address:104.36.82.60:4370): protocol: write tcp 5.161.89.252:37860->104.36.82.60:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456649108 (202h55m1.187916728s) [noise=true , address=104.36.82.5:4857 , piece_id=ZXKQEZHM4OEOQDOSUZN7S5BJTZRNP7INF2LCXHMTWXSUFQ7K6V7Q, seg_pos={0 0}, upload=211119]: Done uploading piece. err: ecclient: upload failed (node:1qjxNyfMA6ZjUurQZty2L8U5SUGo6wcLfCb3KVbmWn3ZaDXzQz, address:104.36.82.5:4857): protocol: write tcp 5.161.89.252:20634->104.36.82.5:4857: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:21:20.456668013 (202h55m1.187935644s) [seg_pos={0 0}, upload=211119]: Done waiting for segment. successful: 8 optimal: 14 errs: <nil> Aug 17 04:21:20.480910798 (202h55m1.212178428s) [upload=211119]: Done uploading segments... Aug 17 04:21:20.480922610 (202h55m1.212190241s) [upload=211119]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 8) Duration: 1m0.025791818s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000546282s FAIL (2025-08-17 04:21:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000566791s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.022034787s FAIL (2025-08-17 04:20:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== Aug 17 04:19:58.525664604 (202h53m39.256932234s) [upload=211113]: Starting upload Aug 17 04:19:58.525708267 (202h53m39.256975886s) [upload=211113]: Done starting upload Aug 17 04:19:58.525806231 (202h53m39.257073861s) [upload=211113]: Uploading segments... Aug 17 04:19:58.525959540 (202h53m39.25722717s) [upload=211113]: Got next segment. Inline: false Aug 17 04:19:58.525997781 (202h53m39.257265411s) [upload=211113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:19:58.526001568 (202h53m39.257269198s) [upload=211113]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:19:58.555374934 (202h53m39.286642554s) [seg_pos={0 0}, upload=211113]: Begin upload segment... Aug 17 04:19:58.555415632 (202h53m39.286683262s) [seg_pos={0 0}, upload=211113]: Done begin upload segment. Aug 17 04:19:58.555435578 (202h53m39.286703199s) [noise=true , address=104.36.82.44:4873 , piece_id=IQNSO55PT2JISSOPAJOM5HB4X7VJZ26OBNEY2NQZIP6TOFQ2XMSA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.555466797 (202h53m39.286734427s) [noise=true , address=77.90.31.11:4582 , piece_id=NHDCZVLBNK6AWHEC2SN5GXR4C5GIVBOHJEAOXAQJARTWGDECGFLQ, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.555722429 (202h53m39.286990059s) [noise=true , address=104.36.82.58:4280 , piece_id=IQRO2H4TG2LATIMJHZAY3WTIBUXEVQHJZOCCSO2ENLZDSUCSNAXA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.555792270 (202h53m39.28705989s) [noise=true , address=79.127.201.215:4360 , piece_id=MCF73MC5ZXPEC7EJGMMCUPENNG7WMHLT2TMKBBOS2QOSMDQAZ35Q, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.555998409 (202h53m39.287266039s) [noise=true , address=77.90.31.19:4866 , piece_id=E6FQSOBNWNF4CFO7YJHVOKJ2DEUJHNWCZK7JRSVAZIWW6R232WOA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556069251 (202h53m39.287336881s) [noise=true , address=104.36.82.22:4543 , piece_id=6N2ZBFRYJQSMBCN6USWPXLRUOCXQKDQHG7P3A3TYAGATJM37Y2OA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556217200 (202h53m39.28748483s) [noise=true , address=156.146.43.225:4468 , piece_id=OJXN5QAYL3BIBOK6AMM5FSHLWACDVSV3GDRLPROYGZXFPL5VFNIQ, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556290047 (202h53m39.287557678s) [noise=true , address=77.90.31.18:4814 , piece_id=4B4YBG6OMTGVSMSCPB7ZKX6EV6WCAFBYLNYBLQKYOPSB2EFNOSDA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556460889 (202h53m39.287728509s) [noise=true , address=79.127.219.48:4101 , piece_id=FYW3DMR67D4QBNMFPQFOG2JL5QZR7WTRF52I5NWQYPZ2SQZJIN5A, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556514199 (202h53m39.287781829s) [noise=true , address=45.91.116.14:4203 , piece_id=LBCXXXG2F5D7YD2TIN7DRYNKHU43NASORP4DTS3F3JN7AVHUZKQA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556686483 (202h53m39.287954103s) [noise=true , address=104.36.82.19:4436 , piece_id=YJEAPLTWSGIFNQG445JXDGSZKUQWYNFLJJRXVS2R4TMI27LPAAKA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556726259 (202h53m39.287993878s) [noise=true , address=45.91.116.11:4127 , piece_id=PBXKPHJZS4M4BG7AGCMCD4TCQOPP3U3Z3UHTTSWCX7QRTULPT2LA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556925844 (202h53m39.288193464s) [noise=true , address=45.91.116.7:4027 , piece_id=O4DX6GTDNVJIASDQ5363DXOKLZECS5RZQNVSRADYLZGAIUR4HKIA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.556953055 (202h53m39.288220685s) [noise=true , address=104.36.82.60:4348 , piece_id=7IWTVGKBZU6ZPMOKFR65WSJ5IPC5HGCOZHAC4TQISKUJ2XQ6NK2A, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.557181655 (202h53m39.288449286s) [noise=true , address=79.127.220.101:4062 , piece_id=YGAQ3V3Y4ZMOPXNC4VLJ7GDVU5ATQLY5UD4A627W4YNBMZ6GFMGA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.557414954 (202h53m39.288682574s) [noise=true , address=104.36.82.18:4380 , piece_id=KBCO75EZCOZD4L4JTOXXS65ZDANGXSMHOOVGT73HWGLEU4YZIQMA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.557629739 (202h53m39.288897359s) [noise=true , address=79.127.201.215:4357 , piece_id=RLAP57D74DUHS6HQYPLKMO5LKUN7PC33LQQFFK24PHWN4JA3Q3UA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.557945884 (202h53m39.289213504s) [noise=true , address=66.23.193.175:4558 , piece_id=KC7X6LQ2PS2V246VC3PHTUE3XRZX7IARLQKEXUGDGXXDNVWJASIA, seg_pos={0 0}, upload=211113]: Uploading piece... Aug 17 04:19:58.632732718 (202h53m39.364000348s) [upload=211113]: Next returned nil segment Aug 17 04:19:58.632744140 (202h53m39.36401176s) [upload=211113]: Waiting for error group managing segments... Aug 17 04:19:58.640857181 (202h53m39.372124832s) [noise=true , address=45.91.116.7:4027 , piece_id=O4DX6GTDNVJIASDQ5363DXOKLZECS5RZQNVSRADYLZGAIUR4HKIA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.641803424 (202h53m39.373071054s) [noise=true , address=79.127.219.48:4101 , piece_id=FYW3DMR67D4QBNMFPQFOG2JL5QZR7WTRF52I5NWQYPZ2SQZJIN5A, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.642078962 (202h53m39.373346612s) [noise=true , address=45.91.116.11:4127 , piece_id=PBXKPHJZS4M4BG7AGCMCD4TCQOPP3U3Z3UHTTSWCX7QRTULPT2LA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.642391500 (202h53m39.37365913s) [noise=true , address=77.90.31.11:4582 , piece_id=NHDCZVLBNK6AWHEC2SN5GXR4C5GIVBOHJEAOXAQJARTWGDECGFLQ, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.672031167 (202h53m39.403298816s) [noise=true , address=45.91.116.14:4203 , piece_id=LBCXXXG2F5D7YD2TIN7DRYNKHU43NASORP4DTS3F3JN7AVHUZKQA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.686973675 (202h53m39.418241316s) [noise=true , address=77.90.31.18:4814 , piece_id=4B4YBG6OMTGVSMSCPB7ZKX6EV6WCAFBYLNYBLQKYOPSB2EFNOSDA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.731085955 (202h53m39.462353585s) [noise=true , address=79.127.220.101:4062 , piece_id=YGAQ3V3Y4ZMOPXNC4VLJ7GDVU5ATQLY5UD4A627W4YNBMZ6GFMGA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.915960686 (202h53m39.647228307s) [noise=true , address=156.146.43.225:4468 , piece_id=OJXN5QAYL3BIBOK6AMM5FSHLWACDVSV3GDRLPROYGZXFPL5VFNIQ, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:58.919779365 (202h53m39.651046985s) [noise=true , address=79.127.201.215:4360 , piece_id=MCF73MC5ZXPEC7EJGMMCUPENNG7WMHLT2TMKBBOS2QOSMDQAZ35Q, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:59.074887261 (202h53m39.806154921s) [noise=true , address=77.90.31.19:4866 , piece_id=E6FQSOBNWNF4CFO7YJHVOKJ2DEUJHNWCZK7JRSVAZIWW6R232WOA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:59.153539802 (202h53m39.884807453s) [noise=true , address=66.23.193.175:4558 , piece_id=KC7X6LQ2PS2V246VC3PHTUE3XRZX7IARLQKEXUGDGXXDNVWJASIA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:19:59.576260912 (202h53m40.307528572s) [noise=true , address=79.127.201.215:4357 , piece_id=RLAP57D74DUHS6HQYPLKMO5LKUN7PC33LQQFFK24PHWN4JA3Q3UA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: <nil> Aug 17 04:20:58.513051975 (202h54m39.244319626s) [noise=true , address=104.36.82.44:4873 , piece_id=IQNSO55PT2JISSOPAJOM5HB4X7VJZ26OBNEY2NQZIP6TOFQ2XMSA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: ecclient: upload failed (node:12b5GdYtmYz9D1QW2z78W3Z2fWvMJGtrvrMJ1njbcqbtCDUdwHo, address:104.36.82.44:4873): protocol: rpc: dial tcp 104.36.82.44:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:58.513122428 (202h54m39.244390069s) [noise=true , address=104.36.82.60:4348 , piece_id=7IWTVGKBZU6ZPMOKFR65WSJ5IPC5HGCOZHAC4TQISKUJ2XQ6NK2A, seg_pos={0 0}, upload=211113]: Done uploading piece. err: ecclient: upload failed (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx, address:104.36.82.60:4348): protocol: rpc: dial tcp 104.36.82.60:4348: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:58.513181409 (202h54m39.244449049s) [noise=true , address=104.36.82.18:4380 , piece_id=KBCO75EZCOZD4L4JTOXXS65ZDANGXSMHOOVGT73HWGLEU4YZIQMA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: ecclient: upload failed (node:1Ebyj96KWVGKwgzRjkAWc9ktcaRW5xqq1bJLEPxfepNvzWvyWN, address:104.36.82.18:4380): protocol: rpc: dial tcp 104.36.82.18:4380: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:58.513227417 (202h54m39.244495067s) [noise=true , address=104.36.82.58:4280 , piece_id=IQRO2H4TG2LATIMJHZAY3WTIBUXEVQHJZOCCSO2ENLZDSUCSNAXA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: ecclient: upload failed (node:1fgHaEFKK4CeGg4x3vywTuYfT3kmac31xqg2QpsahQNXmvjhW8, address:104.36.82.58:4280): protocol: rpc: dial tcp 104.36.82.58:4280: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:58.513250349 (202h54m39.24451799s) [noise=true , address=104.36.82.19:4436 , piece_id=YJEAPLTWSGIFNQG445JXDGSZKUQWYNFLJJRXVS2R4TMI27LPAAKA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:58.513283662 (202h54m39.244551312s) [noise=true , address=104.36.82.22:4543 , piece_id=6N2ZBFRYJQSMBCN6USWPXLRUOCXQKDQHG7P3A3TYAGATJM37Y2OA, seg_pos={0 0}, upload=211113]: Done uploading piece. err: ecclient: upload failed (node:19iUYyo8cGDLGDnvdNo2Qfj1K6tp6EJWF5U6rXD4gJ7Hv7Gsch, address:104.36.82.22:4543): protocol: write tcp 5.161.89.252:60774->104.36.82.22:4543: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:58.513321143 (202h54m39.244588792s) [seg_pos={0 0}, upload=211113]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 17 04:20:58.533017701 (202h54m39.264285341s) [upload=211113]: Done uploading segments... Aug 17 04:20:58.533037498 (202h54m39.264305138s) [upload=211113]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.022054934s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.029189521s FAIL (2025-08-17 04:20:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 04:20:08.668480323 (202h53m49.399747974s) [upload=211116]: Starting upload Aug 17 04:20:08.668575673 (202h53m49.399843324s) [upload=211116]: Done starting upload Aug 17 04:20:08.668654241 (202h53m49.399921901s) [upload=211116]: Uploading segments... Aug 17 04:20:08.668999171 (202h53m49.40026682s) [upload=211116]: Got next segment. Inline: false Aug 17 04:20:08.669063211 (202h53m49.400330861s) [upload=211116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:20:08.669071997 (202h53m49.400339648s) [upload=211116]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:20:08.751335957 (202h53m49.482603607s) [seg_pos={0 0}, upload=211116]: Begin upload segment... Aug 17 04:20:08.751381943 (202h53m49.482649594s) [seg_pos={0 0}, upload=211116]: Done begin upload segment. Aug 17 04:20:08.751412360 (202h53m49.482680011s) [noise=true , address=104.36.82.16:4294 , piece_id=46WLYISEXMTLMOVRQI6HXQKGADP4WOQS62X7YU2VKSE4DFZGYQPA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.751497891 (202h53m49.482765542s) [noise=true , address=45.91.116.13:4153 , piece_id=6VEHPIU7IYBEFTCWH7RJSTPCUAHIICLSCND2R4MMJPGG4HH4FPTQ, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.751940515 (202h53m49.483208165s) [noise=true , address=104.36.82.8:4969 , piece_id=5QIDG45PWF4YZBK4RN2HA2RCU27FIWGDF3UIOGWHMNE3DFCMRUPA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.751990979 (202h53m49.48325863s) [noise=true , address=104.36.82.14:4223 , piece_id=EUUBRTRMCXAGTSIXCUO2ULVQK2GVTMSFNUXUBOPAMCBORNI7DQMA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.752426900 (202h53m49.48369455s) [noise=true , address=45.91.116.10:4082 , piece_id=YQN5MA5BWCKM6WUWQ4YVITI623UWSHO6AY7VN7PW7KQK2FMM2HZQ, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.752439404 (202h53m49.483707064s) [noise=true , address=45.91.116.11:4119 , piece_id=R2KPHORPUUH5545TUEF4HHJYBRLLU7CHJAQJBYAVB5YE55MIDKTQ, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.752808278 (202h53m49.484075938s) [noise=true , address=104.36.82.14:4221 , piece_id=LCXO4KZYL6P6JRST36CV4LTUPGDNVDJHYELXM3EIDR3IGPMBA7JA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.752824890 (202h53m49.484092539s) [noise=true , address=104.36.82.6:4875 , piece_id=WS2VS2B3XSC4VAYSQXREYJFA3S6BE32SQA6EEUWURXET5ZPMPYZA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.753226175 (202h53m49.484493826s) [noise=true , address=45.91.116.10:4081 , piece_id=7JVHZEY2FQDRO2FZO63KJFGXGGROFL2PMJVCPDWBUY4RSX2LKRFQ, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.753239180 (202h53m49.484506839s) [noise=true , address=45.91.116.6:4974 , piece_id=COTUQFN73AIQZYBSP65B6CKZVOXVUIKTLY6VKGZA6FWV47LUA27A, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.753627712 (202h53m49.484895361s) [noise=true , address=45.91.116.9:4261 , piece_id=7DHDWMUGMK74BRVXQUYO56STOVYICPKKXKNHOMAZOJU3QGQTP6KA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.753658108 (202h53m49.484925768s) [noise=true , address=104.36.82.11:4113 , piece_id=3P6YIF5N6J7XMDLPXIK3W2EWSYTB4NJZBIVRC7GGPJM364IZKN6A, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.754001404 (202h53m49.485269054s) [noise=true , address=45.91.116.12:4227 , piece_id=OMHP2ZRA6BLKD65S3JRPRHPFDXMXVCIONONUPZTRGWO253X4BTNA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.754080213 (202h53m49.485347863s) [noise=true , address=45.91.116.14:4189 , piece_id=MOLTUP43V5XKXNA32UTI4PTYBV66LVIS75WBRLZSBN6IDMA7U4SA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.754358297 (202h53m49.485625946s) [noise=true , address=45.91.116.8:4047 , piece_id=SWH6ONYWFHKM4TRNMXHSADIHSTUCL4LBDPGKGSVB2N5CR24JZHIA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.754726389 (202h53m49.485994039s) [noise=true , address=104.36.82.8:4971 , piece_id=22WNKAJKB3TAJG3LRZV5CO5TSDRNLP2L4VZR7SME6BNINPRUEYMA, seg_pos={0 0}, upload=211116]: Uploading piece... Aug 17 04:20:08.861576481 (202h53m49.592844131s) [upload=211116]: Next returned nil segment Aug 17 04:20:08.861592562 (202h53m49.592860222s) [upload=211116]: Waiting for error group managing segments... Aug 17 04:20:08.871671024 (202h53m49.602938684s) [noise=true , address=45.91.116.9:4261 , piece_id=7DHDWMUGMK74BRVXQUYO56STOVYICPKKXKNHOMAZOJU3QGQTP6KA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.872022145 (202h53m49.603289795s) [noise=true , address=45.91.116.10:4082 , piece_id=YQN5MA5BWCKM6WUWQ4YVITI623UWSHO6AY7VN7PW7KQK2FMM2HZQ, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.872559316 (202h53m49.603826966s) [noise=true , address=45.91.116.13:4153 , piece_id=6VEHPIU7IYBEFTCWH7RJSTPCUAHIICLSCND2R4MMJPGG4HH4FPTQ, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.872871533 (202h53m49.604139184s) [noise=true , address=45.91.116.6:4974 , piece_id=COTUQFN73AIQZYBSP65B6CKZVOXVUIKTLY6VKGZA6FWV47LUA27A, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.873278900 (202h53m49.604546551s) [noise=true , address=45.91.116.14:4189 , piece_id=MOLTUP43V5XKXNA32UTI4PTYBV66LVIS75WBRLZSBN6IDMA7U4SA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.873923144 (202h53m49.605190794s) [noise=true , address=45.91.116.8:4047 , piece_id=SWH6ONYWFHKM4TRNMXHSADIHSTUCL4LBDPGKGSVB2N5CR24JZHIA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.874000800 (202h53m49.605268449s) [noise=true , address=45.91.116.12:4227 , piece_id=OMHP2ZRA6BLKD65S3JRPRHPFDXMXVCIONONUPZTRGWO253X4BTNA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.874603183 (202h53m49.605870834s) [noise=true , address=45.91.116.10:4081 , piece_id=7JVHZEY2FQDRO2FZO63KJFGXGGROFL2PMJVCPDWBUY4RSX2LKRFQ, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:08.875079651 (202h53m49.606347301s) [noise=true , address=45.91.116.11:4119 , piece_id=R2KPHORPUUH5545TUEF4HHJYBRLLU7CHJAQJBYAVB5YE55MIDKTQ, seg_pos={0 0}, upload=211116]: Done uploading piece. err: <nil> Aug 17 04:20:38.655786199 (202h54m19.387053849s) [noise=true , address=104.36.82.14:4223 , piece_id=EUUBRTRMCXAGTSIXCUO2ULVQK2GVTMSFNUXUBOPAMCBORNI7DQMA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: rpc: dial tcp 104.36.82.14:4223: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.655890404 (202h54m19.387158054s) [noise=true , address=104.36.82.6:4875 , piece_id=WS2VS2B3XSC4VAYSQXREYJFA3S6BE32SQA6EEUWURXET5ZPMPYZA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: write tcp 5.161.89.252:59142->104.36.82.6:4875: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.656047259 (202h54m19.387314889s) [noise=true , address=104.36.82.11:4113 , piece_id=3P6YIF5N6J7XMDLPXIK3W2EWSYTB4NJZBIVRC7GGPJM364IZKN6A, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.656120849 (202h54m19.387388479s) [noise=true , address=104.36.82.8:4969 , piece_id=5QIDG45PWF4YZBK4RN2HA2RCU27FIWGDF3UIOGWHMNE3DFCMRUPA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: rpc: dial tcp 104.36.82.8:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.656159010 (202h54m19.38742663s) [noise=true , address=104.36.82.16:4294 , piece_id=46WLYISEXMTLMOVRQI6HXQKGADP4WOQS62X7YU2VKSE4DFZGYQPA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: rpc: dial tcp 104.36.82.16:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.656174489 (202h54m19.387442139s) [noise=true , address=104.36.82.8:4971 , piece_id=22WNKAJKB3TAJG3LRZV5CO5TSDRNLP2L4VZR7SME6BNINPRUEYMA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 5.161.89.252:45520->104.36.82.8:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.656227960 (202h54m19.387495611s) [noise=true , address=104.36.82.14:4221 , piece_id=LCXO4KZYL6P6JRST36CV4LTUPGDNVDJHYELXM3EIDR3IGPMBA7JA, seg_pos={0 0}, upload=211116]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:104.36.82.14:4221): protocol: write tcp 5.161.89.252:30652->104.36.82.14:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:38.656270871 (202h54m19.387538521s) [seg_pos={0 0}, upload=211116]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 04:20:38.683245651 (202h54m19.414513321s) [upload=211116]: Done uploading segments... Aug 17 04:20:38.683274505 (202h54m19.414542125s) [upload=211116]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.02920474s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.018084101s FAIL (2025-08-17 04:20:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 11108]: Starting upload Aug 17 04:19:38.569775808 (202h53m19.301043428s) [upload=211108]: Done starting upload Aug 17 04:19:38.569926301 (202h53m19.301193931s) [upload=211108]: Uploading segments... Aug 17 04:19:38.570142017 (202h53m19.301409638s) [upload=211108]: Got next segment. Inline: false Aug 17 04:19:38.570194215 (202h53m19.301461835s) [upload=211108]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:19:38.570198573 (202h53m19.301466194s) [upload=211108]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:19:38.592499273 (202h53m19.323766903s) [seg_pos={0 0}, upload=211108]: Begin upload segment... Aug 17 04:19:38.592538958 (202h53m19.323806579s) [seg_pos={0 0}, upload=211108]: Done begin upload segment. Aug 17 04:19:38.592557103 (202h53m19.323824723s) [noise=true , address=77.90.31.10:4517 , piece_id=4FQ3CRGDSTMRGIT4XUDRIKRXSP6ZY4N4LAHMGU22Z63KTOZZYXNQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.592576098 (202h53m19.323843728s) [noise=true , address=104.36.82.10:4077 , piece_id=WGMQBEMCUH6LHRTLBUREF2DFSUVVBY3RDP4SHTFFRVHKVKRSBBHQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.592839635 (202h53m19.324107265s) [noise=true , address=45.91.116.6:4974 , piece_id=WT7QKBN3647LPTRTE2ULTUWKUWOAIRUIICJKWMVBJ4XGAK6NOIVA, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.592862237 (202h53m19.324129867s) [noise=true , address=45.91.116.5:4298 , piece_id=4QBIYJBUZWTOAGHOAXYWHLSVBU3M25DCLYDSIVFLJ5P2Q3HV4BOQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.593074846 (202h53m19.324342477s) [noise=true , address=104.36.82.14:4222 , piece_id=IMOGXJLXFX64LVDNKUKXZVGVKRZ2UPXQKUWJJBQOT5M36DIWLAJQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.593292687 (202h53m19.324560317s) [noise=true , address=77.90.31.16:4735 , piece_id=MRUR6OQGFKL2ZYKUOMM5IP4Z7QEMBW4TKJAUAGLFM2QNQL2AQTCA, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.593498625 (202h53m19.324766245s) [noise=true , address=45.91.116.9:4263 , piece_id=WA3FA6PP53PID6GGE7E7RQCGAKLOZDT3YXSHRNXIBOZCKFEQLE5A, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.593740801 (202h53m19.325008421s) [noise=true , address=104.36.82.11:4115 , piece_id=YIPCV3RCVNO6K55GN6Y6GQ3SHXURB5JTHFSAO5IORG7UAV3RIL4Q, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.593985000 (202h53m19.325252631s) [noise=true , address=45.91.116.4:4938 , piece_id=6IHV2BWHWOR4XDXBGADUOKQYALYO7TUA7OJKTZYHOARU467HB7CA, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.594205345 (202h53m19.325472965s) [noise=true , address=45.91.116.11:4118 , piece_id=Q7R52VZH7TQRRNDWD2V23MDNSG6G4TNRAHIIQ3PZMZXU65JZPEGQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.594421452 (202h53m19.325689082s) [noise=true , address=104.36.82.6:4875 , piece_id=UXXP472Y4D3PB6NJWTIYGFAS2RIT5YLGRCAWYFLASCVZVRM2W36A, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.594628772 (202h53m19.325896402s) [noise=true , address=45.91.116.7:4010 , piece_id=CQD7PTVWPTA6TNZOQUOMFLNGHDATLERCR33PJYISQCYM4TR2WIXQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.594835752 (202h53m19.326103382s) [noise=true , address=104.36.82.9:4006 , piece_id=DXUNQRCSXFNDM7JJPAOABABCNZVLNTBNRP3UVVAX3A6BHLVMG6YQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.595051859 (202h53m19.326319478s) [noise=true , address=104.36.82.6:4876 , piece_id=AP3UDDSK2LMDP6HBPD3HGFCVEPRSJTIO5BZBYD7AUMOVSFGNQ3KA, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.595267105 (202h53m19.326534735s) [noise=true , address=104.36.82.13:4187 , piece_id=LQ3DU6LVRXBNO3VJUNAWWX7L76I3S7VWZAJI6TSLDOIV6YZFIHCQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.595838119 (202h53m19.327105749s) [noise=true , address=104.36.82.13:4186 , piece_id=6CRE3L3J4QGXK524M44LDJXKDFASVQ3RBVOVDV4CMWQWES4QKLXQ, seg_pos={0 0}, upload=211108]: Uploading piece... Aug 17 04:19:38.660616673 (202h53m19.391884303s) [upload=211108]: Next returned nil segment Aug 17 04:19:38.660627033 (202h53m19.391894653s) [upload=211108]: Waiting for error group managing segments... Aug 17 04:19:38.668886010 (202h53m19.400153629s) [noise=true , address=45.91.116.11:4118 , piece_id=Q7R52VZH7TQRRNDWD2V23MDNSG6G4TNRAHIIQ3PZMZXU65JZPEGQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.669149205 (202h53m19.400416825s) [noise=true , address=45.91.116.7:4010 , piece_id=CQD7PTVWPTA6TNZOQUOMFLNGHDATLERCR33PJYISQCYM4TR2WIXQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.669222453 (202h53m19.400490074s) [noise=true , address=77.90.31.10:4517 , piece_id=4FQ3CRGDSTMRGIT4XUDRIKRXSP6ZY4N4LAHMGU22Z63KTOZZYXNQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.669435554 (202h53m19.400703174s) [noise=true , address=45.91.116.5:4298 , piece_id=4QBIYJBUZWTOAGHOAXYWHLSVBU3M25DCLYDSIVFLJ5P2Q3HV4BOQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.670303969 (202h53m19.401571589s) [noise=true , address=45.91.116.6:4974 , piece_id=WT7QKBN3647LPTRTE2ULTUWKUWOAIRUIICJKWMVBJ4XGAK6NOIVA, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.671128812 (202h53m19.402396432s) [noise=true , address=77.90.31.16:4735 , piece_id=MRUR6OQGFKL2ZYKUOMM5IP4Z7QEMBW4TKJAUAGLFM2QNQL2AQTCA, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.674100035 (202h53m19.405367665s) [noise=true , address=45.91.116.4:4938 , piece_id=6IHV2BWHWOR4XDXBGADUOKQYALYO7TUA7OJKTZYHOARU467HB7CA, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:19:38.687265128 (202h53m19.418532777s) [noise=true , address=45.91.116.9:4263 , piece_id=WA3FA6PP53PID6GGE7E7RQCGAKLOZDT3YXSHRNXIBOZCKFEQLE5A, seg_pos={0 0}, upload=211108]: Done uploading piece. err: <nil> Aug 17 04:20:08.555437423 (202h53m49.286705083s) [noise=true , address=104.36.82.10:4077 , piece_id=WGMQBEMCUH6LHRTLBUREF2DFSUVVBY3RDP4SHTFFRVHKVKRSBBHQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:104.36.82.10:4077): protocol: rpc: dial tcp 104.36.82.10:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555507174 (202h53m49.286774824s) [noise=true , address=104.36.82.13:4186 , piece_id=6CRE3L3J4QGXK524M44LDJXKDFASVQ3RBVOVDV4CMWQWES4QKLXQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: rpc: dial tcp 104.36.82.13:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555561297 (202h53m49.286828946s) [noise=true , address=104.36.82.14:4222 , piece_id=IMOGXJLXFX64LVDNKUKXZVGVKRZ2UPXQKUWJJBQOT5M36DIWLAJQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555616740 (202h53m49.2868844s) [noise=true , address=104.36.82.11:4115 , piece_id=YIPCV3RCVNO6K55GN6Y6GQ3SHXURB5JTHFSAO5IORG7UAV3RIL4Q, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:104.36.82.11:4115): protocol: rpc: dial tcp 104.36.82.11:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555681051 (202h53m49.286948701s) [noise=true , address=104.36.82.9:4006 , piece_id=DXUNQRCSXFNDM7JJPAOABABCNZVLNTBNRP3UVVAX3A6BHLVMG6YQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: rpc: dial tcp 104.36.82.9:4006: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555695118 (202h53m49.286962768s) [noise=true , address=104.36.82.6:4876 , piece_id=AP3UDDSK2LMDP6HBPD3HGFCVEPRSJTIO5BZBYD7AUMOVSFGNQ3KA, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: rpc: dial tcp 104.36.82.6:4876: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555723431 (202h53m49.286991082s) [noise=true , address=104.36.82.13:4187 , piece_id=LQ3DU6LVRXBNO3VJUNAWWX7L76I3S7VWZAJI6TSLDOIV6YZFIHCQ, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:104.36.82.13:4187): protocol: rpc: dial tcp 104.36.82.13:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555775259 (202h53m49.287042909s) [noise=true , address=104.36.82.6:4875 , piece_id=UXXP472Y4D3PB6NJWTIYGFAS2RIT5YLGRCAWYFLASCVZVRM2W36A, seg_pos={0 0}, upload=211108]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:104.36.82.6:4875): protocol: write tcp 5.161.89.252:58258->104.36.82.6:4875: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:20:08.555812428 (202h53m49.287080079s) [seg_pos={0 0}, upload=211108]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Aug 17 04:20:08.571360279 (202h53m49.302627928s) [upload=211108]: Done uploading segments... Aug 17 04:20:08.571385366 (202h53m49.302653015s) [upload=211108]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.018107454s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 145.02326ms FAIL (2025-08-17 04:05:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 145.037457ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 460.833278ms FAIL (2025-08-17 04:04:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 460.860489ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 523.452005ms FAIL (2025-08-17 04:03:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 523.473875ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 149.027762ms FAIL (2025-08-17 04:03:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 149.049653ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 153.571334ms FAIL (2025-08-17 04:02:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 153.608545ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 134.064859ms FAIL (2025-08-17 04:01:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 134.151ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 197.418601ms FAIL (2025-08-17 04:00:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 197.43867ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 148.21614ms FAIL (2025-08-17 03:59:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 148.239542ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 129.708028ms FAIL (2025-08-17 03:58:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 129.736561ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 218.178795ms FAIL (2025-08-17 03:57:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 218.201777ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 130.045113ms FAIL (2025-08-17 03:56:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 130.098824ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 125.450212ms FAIL (2025-08-17 03:55:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 125.469058ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 148.007539ms FAIL (2025-08-17 03:54:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 148.02928ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 128.369939ms FAIL (2025-08-17 03:54:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 128.387382ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 117.548796ms FAIL (2025-08-17 03:53:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 117.569926ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 135.776353ms FAIL (2025-08-17 03:52:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 135.800509ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 137.834849ms FAIL (2025-08-17 03:51:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 137.865537ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 118.297617ms FAIL (2025-08-17 03:51:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 118.323284ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 125.782921ms FAIL (2025-08-17 03:50:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 125.802818ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 139.489828ms FAIL (2025-08-17 03:49:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 139.510867ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 147.957848ms FAIL (2025-08-17 03:48:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 147.995507ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 399.716268ms FAIL (2025-08-17 03:47:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 399.734011ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 141.146577ms FAIL (2025-08-17 03:47:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 03:22:01.047215101 (201h55m41.778482731s) [upload=210114]: Starting upload Aug 17 03:22:01.047270776 (201h55m41.778538395s) [upload=210114]: Done starting upload Aug 17 03:22:01.047321601 (201h55m41.778589231s) [upload=210114]: Uploading segments... Aug 17 03:22:01.047352319 (201h55m41.778619939s) [upload=210114]: Got next segment. Inline: true Aug 17 03:22:01.047356457 (201h55m41.778624087s) [upload=210114]: Waiting for error group managing segments... Aug 17 03:22:01.047390441 (201h55m41.77865806s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 03:22:01.047394689 (201h55m41.778662319s) [upload=210114]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 03:22:01.047397123 (201h55m41.778664753s) [upload=210114]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 03:22:01.120366258 (201h55m41.851633889s) [upload=210114]: Done uploading segments... Aug 17 03:22:01.120384863 (201h55m41.851652533s) [upload=210114]: Upload finished. err: metaclient: Too Many Requests Duration:* 141.171674ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)