Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X...X.................] duration: 406.902837ms FAIL (2025-08-08 08:02:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 406.930588ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X..............X................] duration: 154.313443ms FAIL (2025-08-08 08:02:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 154.336957ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X..............X...............] duration: 497.16484ms FAIL (2025-08-08 08:01:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 497.193874ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X...X................] duration: 334.357482ms FAIL (2025-08-08 08:01:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 334.377611ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X...X...............] duration: 586.174567ms FAIL (2025-08-08 08:01:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 586.198903ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X..............X..............] duration: 460.929753ms FAIL (2025-08-08 08:01:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 460.958766ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X...X..............] duration: 365.8722ms FAIL (2025-08-08 08:00:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 365.892858ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..............X.............] duration: 142.29583ms FAIL (2025-08-08 07:59:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 142.34364ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X...X.............] duration: 803.489566ms FAIL (2025-08-08 07:59:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 803.522587ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X...X............] duration: 368.310937ms FAIL (2025-08-08 07:59:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 368.332698ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X..............X............] duration: 155.746542ms FAIL (2025-08-08 07:59:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 155.773641ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X..............X...........] duration: 198.433582ms FAIL (2025-08-08 07:58:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 198.457527ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X...X...........] duration: 423.044543ms FAIL (2025-08-08 07:58:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 423.066153ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X..............X..........] duration: 198.012786ms FAIL (2025-08-08 07:57:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 198.051749ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X..............X.........] duration: 262.319913ms FAIL (2025-08-08 07:57:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 262.346181ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X...X..........] duration: 388.466456ms FAIL (2025-08-08 07:57:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 388.49568ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X..............X........] duration: 134.398327ms FAIL (2025-08-08 07:56:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 134.41615ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X...X.........] duration: 375.403856ms FAIL (2025-08-08 07:56:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 375.429804ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X..............X.......] duration: 116.991476ms FAIL (2025-08-08 07:55:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 117.016072ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X...X........] duration: 693.277134ms FAIL (2025-08-08 07:55:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 693.301851ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X..............X......] duration: 134.467084ms FAIL (2025-08-08 07:54:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 134.52391ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X...X.......] duration: 346.26385ms FAIL (2025-08-08 07:54:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 346.293654ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X..............X.....] duration: 176.362327ms FAIL (2025-08-08 07:53:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 176.38491ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X...X......] duration: 367.70579ms FAIL (2025-08-08 07:53:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 367.738971ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..............X....] duration: 147.909633ms FAIL (2025-08-08 07:53:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 147.936013ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X...X.....] duration: 364.086482ms FAIL (2025-08-08 07:53:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 364.110076ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X...X....] duration: 349.281911ms FAIL (2025-08-08 07:52:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 349.301838ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X..............X...] duration: 324.728798ms FAIL (2025-08-08 07:52:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 324.760315ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X..............X..] duration: 156.202922ms FAIL (2025-08-08 07:51:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 156.232258ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X...X...] duration: 370.677495ms FAIL (2025-08-08 07:50:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 370.699916ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X..............X.] duration: 502.779174ms FAIL (2025-08-08 07:50:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 502.804732ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X...X..] duration: 391.129737ms FAIL (2025-08-08 07:49:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 391.15362ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X..............X] duration: 16.373862ms FAIL (2025-08-08 07:49:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== nts... Aug 8 07:42:45.713737125 (15h15m59.300793366s) [upload=15782]: Upload finished. err: <nil> Aug 8 07:43:19.974480154 (15h16m33.561536396s) [upload=15794]: Starting upload Aug 8 07:43:19.974565975 (15h16m33.561622225s) [upload=15794]: Done starting upload Aug 8 07:43:19.974666813 (15h16m33.561723064s) [upload=15794]: Uploading segments... Aug 8 07:43:19.974732767 (15h16m33.561789007s) [upload=15794]: Got next segment. Inline: true Aug 8 07:43:19.974738557 (15h16m33.561794799s) [upload=15794]: Waiting for error group managing segments... Aug 8 07:43:19.974808447 (15h16m33.561864689s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:19.974814609 (15h16m33.56187085s) [upload=15794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:19.974818436 (15h16m33.561874687s) [upload=15794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:20.253892299 (15h16m33.84094854s) [upload=15794]: Done uploading segments... Aug 8 07:43:20.253952420 (15h16m33.841008662s) [upload=15794]: Upload finished. err: <nil> Aug 8 07:43:57.831123452 (15h17m11.418179693s) [upload=15806]: Starting upload Aug 8 07:43:57.831246041 (15h17m11.418302292s) [upload=15806]: Done starting upload Aug 8 07:43:57.831408524 (15h17m11.418464776s) [upload=15806]: Uploading segments... Aug 8 07:43:57.831492601 (15h17m11.418548852s) [upload=15806]: Got next segment. Inline: true Aug 8 07:43:57.831500776 (15h17m11.418557018s) [upload=15806]: Waiting for error group managing segments... Aug 8 07:43:57.831555699 (15h17m11.41861195s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:43:57.831564375 (15h17m11.418620616s) [upload=15806]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:43:57.831570657 (15h17m11.418626908s) [upload=15806]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:43:57.881689019 (15h17m11.46874525s) [upload=15806]: Done uploading segments... Aug 8 07:43:57.881733051 (15h17m11.468789272s) [upload=15806]: Upload finished. err: <nil> Aug 8 07:44:33.628931332 (15h17m47.215987572s) [upload=15813]: Starting upload Aug 8 07:44:33.629002906 (15h17m47.216059147s) [upload=15813]: Done starting upload Aug 8 07:44:33.629070531 (15h17m47.216126773s) [upload=15813]: Uploading segments... Aug 8 07:44:33.629111819 (15h17m47.21616806s) [upload=15813]: Got next segment. Inline: true Aug 8 07:44:33.629117881 (15h17m47.216174121s) [upload=15813]: Waiting for error group managing segments... Aug 8 07:44:33.629163996 (15h17m47.216220237s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:33.629169617 (15h17m47.216225857s) [upload=15813]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:33.629174125 (15h17m47.216230376s) [upload=15813]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:33.686236498 (15h17m47.273292749s) [upload=15813]: Done uploading segments... Aug 8 07:44:33.686255053 (15h17m47.273311294s) [upload=15813]: Upload finished. err: <nil> Aug 8 07:44:55.471623625 (15h18m9.058679867s) [upload=15822]: Starting upload Aug 8 07:44:55.471688186 (15h18m9.058744438s) [upload=15822]: Done starting upload Aug 8 07:44:55.471744130 (15h18m9.058800372s) [upload=15822]: Uploading segments... Aug 8 07:44:55.471799334 (15h18m9.058855575s) [upload=15822]: Got next segment. Inline: true Aug 8 07:44:55.471806668 (15h18m9.058862909s) [upload=15822]: Waiting for error group managing segments... Aug 8 07:44:55.471855819 (15h18m9.05891206s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:44:55.471861650 (15h18m9.058917891s) [upload=15822]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:44:55.471865407 (15h18m9.058921648s) [upload=15822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:44:55.668621888 (15h18m9.255678139s) [upload=15822]: Done uploading segments... Aug 8 07:44:55.668642436 (15h18m9.255698678s) [upload=15822]: Upload finished. err: <nil> Aug 8 07:45:27.402959362 (15h18m40.990015602s) [upload=15829]: Starting upload Aug 8 07:45:27.403025055 (15h18m40.990081295s) [upload=15829]: Done starting upload Aug 8 07:45:27.403107468 (15h18m40.990163709s) [upload=15829]: Uploading segments... Aug 8 07:45:27.403144967 (15h18m40.990201209s) [upload=15829]: Got next segment. Inline: true Aug 8 07:45:27.403150748 (15h18m40.990206989s) [upload=15829]: Waiting for error group managing segments... Aug 8 07:45:27.403195132 (15h18m40.990251372s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:45:27.403200642 (15h18m40.990256883s) [upload=15829]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:45:27.403204299 (15h18m40.99026054s) [upload=15829]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:27.464963266 (15h18m41.052019508s) [upload=15829]: Done uploading segments... Aug 8 07:45:27.464982232 (15h18m41.052038473s) [upload=15829]: Upload finished. err: <nil> Aug 8 07:46:04.417347106 (15h19m18.004403357s) [upload=15840]: Starting upload Aug 8 07:46:04.417413339 (15h19m18.00446959s) [upload=15840]: Done starting upload Aug 8 07:46:04.417487337 (15h19m18.004543578s) [upload=15840]: Uploading segments... Aug 8 07:46:04.417534245 (15h19m18.004590495s) [upload=15840]: Got next segment. Inline: true Aug 8 07:46:04.417541238 (15h19m18.004597499s) [upload=15840]: Waiting for error group managing segments... Aug 8 07:46:04.417592874 (15h19m18.004649124s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:04.417600127 (15h19m18.004656389s) [upload=15840]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:46:04.417604546 (15h19m18.004660797s) [upload=15840]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:46:04.468228743 (15h19m18.055284993s) [upload=15840]: Done uploading segments... Aug 8 07:46:04.468254300 (15h19m18.055310542s) [upload=15840]: Upload finished. err: <nil> Aug 8 07:47:13.348096716 (15h20m26.935152937s) [upload=15858]: Starting upload Aug 8 07:47:13.348153412 (15h20m26.935209633s) [upload=15858]: Done starting upload Aug 8 07:47:13.348212813 (15h20m26.935269024s) [upload=15858]: Uploading segments... Aug 8 07:47:13.348250844 (15h20m26.935307065s) [upload=15858]: Got next segment. Inline: true Aug 8 07:47:13.348255031 (15h20m26.935311242s) [upload=15858]: Waiting for error group managing segments... Aug 8 07:47:13.348288474 (15h20m26.935344685s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:47:13.348293233 (15h20m26.935349454s) [upload=15858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:47:13.348297000 (15h20m26.93535325s) [upload=15858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:47:13.406436854 (15h20m26.993493075s) [upload=15858]: Done uploading segments... Aug 8 07:47:13.406455770 (15h20m26.993511991s) [upload=15858]: Upload finished. err: <nil> Aug 8 07:48:01.965501092 (15h21m15.552557334s) [upload=15875]: Starting upload Aug 8 07:48:01.965571453 (15h21m15.552627695s) [upload=15875]: Done starting upload Aug 8 07:48:01.965628209 (15h21m15.55268445s) [upload=15875]: Uploading segments... Aug 8 07:48:01.965685186 (15h21m15.552741427s) [upload=15875]: Got next segment. Inline: true Aug 8 07:48:01.965692449 (15h21m15.5527487s) [upload=15875]: Waiting for error group managing segments... Aug 8 07:48:01.965743254 (15h21m15.552799495s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:01.965749205 (15h21m15.552805446s) [upload=15875]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:01.965754134 (15h21m15.552810375s) [upload=15875]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:02.020986552 (15h21m15.608042793s) [upload=15875]: Done uploading segments... Aug 8 07:48:02.021010106 (15h21m15.608066347s) [upload=15875]: Upload finished. err: <nil> Aug 8 07:48:45.037162302 (15h21m58.624218533s) [upload=15884]: Starting upload Aug 8 07:48:45.037212516 (15h21m58.624268747s) [upload=15884]: Done starting upload Aug 8 07:48:45.037263130 (15h21m58.624319361s) [upload=15884]: Uploading segments... Aug 8 07:48:45.037303145 (15h21m58.624359377s) [upload=15884]: Got next segment. Inline: true Aug 8 07:48:45.037307644 (15h21m58.624363865s) [upload=15884]: Waiting for error group managing segments... Aug 8 07:48:45.037344523 (15h21m58.624400744s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:45.037349141 (15h21m58.624405362s) [upload=15884]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:48:45.037352057 (15h21m58.624408278s) [upload=15884]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:45.358091403 (15h21m58.945147643s) [upload=15884]: Done uploading segments... Aug 8 07:48:45.358110608 (15h21m58.945166849s) [upload=15884]: Upload finished. err: <nil> Aug 8 07:49:25.751491669 (15h22m39.338547911s) [upload=15896]: Starting upload Aug 8 07:49:25.751590724 (15h22m39.338646965s) [upload=15896]: Done starting upload Aug 8 07:49:25.751673849 (15h22m39.338730091s) [upload=15896]: Uploading segments... Aug 8 07:49:25.751714535 (15h22m39.338770777s) [upload=15896]: Got next segment. Inline: true Aug 8 07:49:25.751720216 (15h22m39.338776457s) [upload=15896]: Waiting for error group managing segments... Aug 8 07:49:25.751764209 (15h22m39.338820449s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:49:25.751769769 (15h22m39.33882602s) [upload=15896]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:49:25.751773475 (15h22m39.338829716s) [upload=15896]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:49:25.754547443 (15h22m39.341603683s) [upload=15896]: Done uploading segments... Aug 8 07:49:25.754560417 (15h22m39.341616668s) [upload=15896]: Upload finished. err: metaclient: Too Many Requests Duration: 16.396262ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X...X.] duration: 417.764699ms FAIL (2025-08-08 07:49:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 417.789264ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X...X] duration: 17.248012ms FAIL (2025-08-08 07:48:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558478434 (15h21m24.145534675s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558682175 (15h21m24.145738415s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.558865587 (15h21m24.145921827s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559147964 (15h21m24.146204215s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559223675 (15h21m24.146279926s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559571845 (15h21m24.146628096s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559582545 (15h21m24.146638787s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.559999063 (15h21m24.147055313s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560030722 (15h21m24.147086973s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560388609 (15h21m24.14744485s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560501089 (15h21m24.14755733s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560783627 (15h21m24.147839868s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.560888993 (15h21m24.147945235s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561172963 (15h21m24.148229214s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561374930 (15h21m24.148431171s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.561522005 (15h21m24.148578246s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Uploading piece... Aug 8 07:48:10.701061446 (15h21m24.288117687s) [upload=15877]: Next returned nil segment Aug 8 07:48:10.701078157 (15h21m24.288134398s) [upload=15877]: Waiting for error group managing segments... Aug 8 07:48:10.712346007 (15h21m24.299402248s) [noise=true , address=45.91.116.13:4160 , piece_id=SYEMBSA3FJSFDNS7NVOP6WHNT5SZSQE3JJWATVUPNDMPOR3WBKBQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.714446768 (15h21m24.301503019s) [noise=true , address=104.36.82.18:4385 , piece_id=GKWAMKIEZX3T4QFTCQXOE62LN6GMLQXQ7L35KZDAL3MCPFA4FCBA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715282398 (15h21m24.302338649s) [noise=true , address=104.36.82.62:4413 , piece_id=6GARUTEVOCXUWBI7QPNDIK5DOMEIFDYU4GRLQIK24F5VJLQNEDCQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715562772 (15h21m24.302619012s) [noise=true , address=104.36.82.8:4974 , piece_id=EIHHWLOQIRDI55ZVKR2YRZI4FJ4V2Q2UM3JRM4EJTD7XKZPRSR5Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.715940857 (15h21m24.302997099s) [noise=true , address=104.36.82.12:4163 , piece_id=C3O2XTUVL52P7KUNVMION2DPZJL6554GAJYUQQUB2ARXKQOW2Y2A, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716415382 (15h21m24.303471624s) [noise=true , address=104.36.82.43:4836 , piece_id=WHJMQSX3AMO5LK6U4S6Q63XKIVAF5C62DGQ5P4XFG45GHF6MLPGA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.716787086 (15h21m24.303843328s) [noise=true , address=104.36.82.37:4687 , piece_id=XQDWCWKLN2NKHF5FWQMVQ6MNWMKVYVHS66SH4SGAHGEYPKROO3SQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717228611 (15h21m24.304284862s) [noise=true , address=104.36.82.59:4334 , piece_id=YKIIOC2SEK3L3NUMK4YHW4N5KLZSNLD4QJKQC2BICEGMFHD35XHA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.717487164 (15h21m24.304543405s) [noise=true , address=104.36.82.46:4481 , piece_id=5NVIAOMDR7HTNOJTFBTZXJT7KTWGTQC7N6I27NQ6TGSXWETXVFJA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718000281 (15h21m24.305056522s) [noise=true , address=104.36.82.44:4872 , piece_id=LRXX4LUFK232KOK6RDB2WQJG6PBHGKOUODLOH5BVLREB7JC2I22Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.718057548 (15h21m24.305113788s) [noise=true , address=104.36.82.52:4110 , piece_id=EUXML3SLDEE2CHO2U464QFDJZ6JIMFNW4U6AOMU3PWPBRWKYQPMA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719189180 (15h21m24.306245421s) [noise=true , address=104.36.82.61:4383 , piece_id=JNKF3PCQ32NNIH2GZNVVG5JD2ROB7HYU3QVZDE32RDGI42TMGEQQ, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719657965 (15h21m24.306714206s) [noise=true , address=104.36.82.35:4613 , piece_id=V2FKAJMML2B234ONO6DTDBMRLCL2VBM5QNH4MZXMKZ6VWATVGX2Q, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719794319 (15h21m24.30685057s) [noise=true , address=104.36.82.9:4014 , piece_id=YAWSG2KYDYZJ7GYOYHAYAIBYZKKF6IXRCOW77IZLXO2WELYAQ2KA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: <nil> Aug 8 07:48:10.719820398 (15h21m24.306876639s) [seg_pos={0 0}, upload=15877]: Segment reached optimal threshold of 14 pieces. Aug 8 07:48:10.719998841 (15h21m24.307055082s) [noise=true , address=104.36.82.13:4192 , piece_id=VGRQOIVL77SRMUWJUW3BQ2BGRR3LIEGTIKBAXPKXAJUH5MTCIYVA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12nKVH5WuEdpXN7Z1rbD6yzv3HF7Fi9Gm5FBbc4gMqj93aFjuN3): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720104438 (15h21m24.30716068s) [noise=true , address=104.36.82.47:4542 , piece_id=EP3FF3PJFO2IRAHIOD2ZVNAJDNBZ2A5IWXMRE2JC5U3XU2IAC2UA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU): protocol: expected piece hash; context canceled; EOF Aug 8 07:48:10.720349365 (15h21m24.307405606s) [noise=true , address=64.49.13.154:4928 , piece_id=KNLHZAZ5PJ52EKVJEDKVTJNAZJUKLWBNQ6DLU634SYDPSDOFXNTA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:54334->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720361257 (15h21m24.307417498s) [noise=true , address=64.49.13.155:4921 , piece_id=IMROAPMZ55WVWF7YBP4VBQLQWYVG4NLYW5CC5V6PEKFFIQ34WCXA, seg_pos={0 0}, upload=15877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:172g6Fonibtkmk7deZRa1fcR8WdxHNfY4oDz9WgdyQifnwKqJ5): protocol: read tcp 5.161.89.252:54174->64.49.13.155:4921: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:48:10.720393558 (15h21m24.307449809s) [seg_pos={0 0}, upload=15877]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:48:10.720457627 (15h21m24.307513868s) [upload=15877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:48:10.720463047 (15h21m24.307519298s) [upload=15877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:48:10.766500413 (15h21m24.353556654s) [upload=15877]: Done uploading segments... Aug 8 07:48:10.766516383 (15h21m24.353572624s) [upload=15877]: Upload finished. err: <nil> Aug 8 07:48:55.848943186 (15h22m9.435999427s) [upload=15886]: Starting upload Aug 8 07:48:55.849011334 (15h22m9.436067575s) [upload=15886]: Done starting upload Aug 8 07:48:55.849136027 (15h22m9.436192238s) [upload=15886]: Uploading segments... Aug 8 07:48:55.849367489 (15h22m9.43642372s) [upload=15886]: Got next segment. Inline: false Aug 8 07:48:55.849417472 (15h22m9.436473703s) [upload=15886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:48:55.849422581 (15h22m9.436478823s) [upload=15886]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:48:55.852372246 (15h22m9.439428467s) [upload=15886]: Next segment err: metaclient: Too Many Requests Aug 8 07:48:55.852399558 (15h22m9.439455768s) [upload=15886]: Done uploading segments... Aug 8 07:48:55.852411209 (15h22m9.43946743s) [upload=15886]: Upload finished. err: metaclient: Too Many Requests Duration: 17.269091ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 469.917498ms FAIL (2025-08-08 07:48:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 469.947935ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 463.568731ms FAIL (2025-08-08 07:48:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 30]: Uploading piece... Aug 8 07:45:29.750686944 (15h18m43.337743185s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.750696903 (15h18m43.337753144s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751077063 (15h18m43.338133314s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751096508 (15h18m43.338152759s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751438256 (15h18m43.338494498s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751454708 (15h18m43.338510948s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751803749 (15h18m43.33886s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751812055 (15h18m43.338868295s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752268797 (15h18m43.339325038s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752632205 (15h18m43.339688446s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753002305 (15h18m43.340058547s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753418863 (15h18m43.340475114s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753859585 (15h18m43.340915837s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754388493 (15h18m43.341444735s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754765928 (15h18m43.341822178s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.755148412 (15h18m43.342204653s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.870594195 (15h18m43.457650436s) [upload=15830]: Next returned nil segment Aug 8 07:45:29.870622687 (15h18m43.457678938s) [upload=15830]: Waiting for error group managing segments... Aug 8 07:45:29.880490816 (15h18m43.467547067s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882136728 (15h18m43.469192969s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882913007 (15h18m43.469969248s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883383075 (15h18m43.470439315s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883463644 (15h18m43.470519886s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884241697 (15h18m43.471297938s) [noise=true , address=104.36.82.61:4375 , piece_id=VH56RRFT6UY2NHKAT2X63KKPPJSY2PF4KFYGFHQSFKIKMP7FY4TA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884323910 (15h18m43.471380151s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884973903 (15h18m43.472030154s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884984272 (15h18m43.472040514s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885552193 (15h18m43.472608444s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885640007 (15h18m43.472696247s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886142264 (15h18m43.473198504s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886250977 (15h18m43.473307217s) [noise=true , address=104.36.82.13:4215 , piece_id=MGZGACY7FQZKFOLCMDD2ABTLYKIVM7DM5XCPBTKRDRQOMLLYWXQA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887026404 (15h18m43.474082645s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887050258 (15h18m43.4741065s) [seg_pos={0 0}, upload=15830]: Segment reached optimal threshold of 14 pieces. Aug 8 07:45:29.887256934 (15h18m43.474313185s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N): protocol: expected piece hash; context canceled; EOF Aug 8 07:45:29.887313620 (15h18m43.474369862s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r5KA1p2BjwSqZvkS5skmU9t8WbF6rFkEqvvACPsfaPFuY6s3p): protocol: read tcp 5.161.89.252:46414->64.49.13.154:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887426782 (15h18m43.474483022s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1soiTQsiMNe5BkNHwwAGykmtP9LtLxncqE7Tv728ak8pjFeshg): protocol: read tcp 5.161.89.252:22516->64.49.13.26:4947: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887480762 (15h18m43.474537003s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: read tcp 5.161.89.252:16684->64.49.13.170:4717: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 8 07:45:29.887515146 (15h18m43.474571387s) [seg_pos={0 0}, upload=15830]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:45:29.887578214 (15h18m43.474634455s) [upload=15830]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:45:29.887584155 (15h18m43.474640396s) [upload=15830]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:30.055211210 (15h18m43.642267511s) [upload=15830]: Done uploading segments... Aug 8 07:45:30.055232540 (15h18m43.642288781s) [upload=15830]: Upload finished. err: <nil> Aug 8 07:46:24.069800557 (15h19m37.656856798s) [upload=15846]: Starting upload Aug 8 07:46:24.069887619 (15h19m37.65694386s) [upload=15846]: Done starting upload Aug 8 07:46:24.070014306 (15h19m37.657070546s) [upload=15846]: Uploading segments... Aug 8 07:46:24.070283849 (15h19m37.65734009s) [upload=15846]: Got next segment. Inline: false Aug 8 07:46:24.070335675 (15h19m37.657391917s) [upload=15846]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:24.070341606 (15h19m37.657397848s) [upload=15846]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:46:24.073193329 (15h19m37.66024957s) [upload=15846]: Next segment err: metaclient: Too Many Requests Aug 8 07:46:24.073210160 (15h19m37.660266402s) [upload=15846]: Done uploading segments... Aug 8 07:46:24.073226000 (15h19m37.660282241s) [upload=15846]: Upload finished. err: metaclient: Too Many Requests Duration: 463.603746ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 168.027ms FAIL (2025-08-08 07:48:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 168.057588ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 661.356416ms FAIL (2025-08-08 07:47:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 30]: Uploading piece... Aug 8 07:45:29.750686944 (15h18m43.337743185s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.750696903 (15h18m43.337753144s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751077063 (15h18m43.338133314s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751096508 (15h18m43.338152759s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751438256 (15h18m43.338494498s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751454708 (15h18m43.338510948s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751803749 (15h18m43.33886s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751812055 (15h18m43.338868295s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752268797 (15h18m43.339325038s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752632205 (15h18m43.339688446s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753002305 (15h18m43.340058547s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753418863 (15h18m43.340475114s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753859585 (15h18m43.340915837s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754388493 (15h18m43.341444735s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754765928 (15h18m43.341822178s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.755148412 (15h18m43.342204653s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.870594195 (15h18m43.457650436s) [upload=15830]: Next returned nil segment Aug 8 07:45:29.870622687 (15h18m43.457678938s) [upload=15830]: Waiting for error group managing segments... Aug 8 07:45:29.880490816 (15h18m43.467547067s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882136728 (15h18m43.469192969s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882913007 (15h18m43.469969248s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883383075 (15h18m43.470439315s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883463644 (15h18m43.470519886s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884241697 (15h18m43.471297938s) [noise=true , address=104.36.82.61:4375 , piece_id=VH56RRFT6UY2NHKAT2X63KKPPJSY2PF4KFYGFHQSFKIKMP7FY4TA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884323910 (15h18m43.471380151s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884973903 (15h18m43.472030154s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884984272 (15h18m43.472040514s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885552193 (15h18m43.472608444s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885640007 (15h18m43.472696247s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886142264 (15h18m43.473198504s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886250977 (15h18m43.473307217s) [noise=true , address=104.36.82.13:4215 , piece_id=MGZGACY7FQZKFOLCMDD2ABTLYKIVM7DM5XCPBTKRDRQOMLLYWXQA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887026404 (15h18m43.474082645s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887050258 (15h18m43.4741065s) [seg_pos={0 0}, upload=15830]: Segment reached optimal threshold of 14 pieces. Aug 8 07:45:29.887256934 (15h18m43.474313185s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N): protocol: expected piece hash; context canceled; EOF Aug 8 07:45:29.887313620 (15h18m43.474369862s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r5KA1p2BjwSqZvkS5skmU9t8WbF6rFkEqvvACPsfaPFuY6s3p): protocol: read tcp 5.161.89.252:46414->64.49.13.154:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887426782 (15h18m43.474483022s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1soiTQsiMNe5BkNHwwAGykmtP9LtLxncqE7Tv728ak8pjFeshg): protocol: read tcp 5.161.89.252:22516->64.49.13.26:4947: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887480762 (15h18m43.474537003s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: read tcp 5.161.89.252:16684->64.49.13.170:4717: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 8 07:45:29.887515146 (15h18m43.474571387s) [seg_pos={0 0}, upload=15830]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:45:29.887578214 (15h18m43.474634455s) [upload=15830]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:45:29.887584155 (15h18m43.474640396s) [upload=15830]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:30.055211210 (15h18m43.642267511s) [upload=15830]: Done uploading segments... Aug 8 07:45:30.055232540 (15h18m43.642288781s) [upload=15830]: Upload finished. err: <nil> Aug 8 07:46:24.069800557 (15h19m37.656856798s) [upload=15846]: Starting upload Aug 8 07:46:24.069887619 (15h19m37.65694386s) [upload=15846]: Done starting upload Aug 8 07:46:24.070014306 (15h19m37.657070546s) [upload=15846]: Uploading segments... Aug 8 07:46:24.070283849 (15h19m37.65734009s) [upload=15846]: Got next segment. Inline: false Aug 8 07:46:24.070335675 (15h19m37.657391917s) [upload=15846]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:24.070341606 (15h19m37.657397848s) [upload=15846]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:46:24.073193329 (15h19m37.66024957s) [upload=15846]: Next segment err: metaclient: Too Many Requests Aug 8 07:46:24.073210160 (15h19m37.660266402s) [upload=15846]: Done uploading segments... Aug 8 07:46:24.073226000 (15h19m37.660282241s) [upload=15846]: Upload finished. err: metaclient: Too Many Requests Duration: 661.383988ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 131.273173ms FAIL (2025-08-08 07:47:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 131.294112ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 624.103088ms FAIL (2025-08-08 07:46:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 30]: Uploading piece... Aug 8 07:45:29.750686944 (15h18m43.337743185s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.750696903 (15h18m43.337753144s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751077063 (15h18m43.338133314s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751096508 (15h18m43.338152759s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751438256 (15h18m43.338494498s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751454708 (15h18m43.338510948s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751803749 (15h18m43.33886s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751812055 (15h18m43.338868295s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752268797 (15h18m43.339325038s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752632205 (15h18m43.339688446s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753002305 (15h18m43.340058547s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753418863 (15h18m43.340475114s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753859585 (15h18m43.340915837s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754388493 (15h18m43.341444735s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754765928 (15h18m43.341822178s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.755148412 (15h18m43.342204653s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.870594195 (15h18m43.457650436s) [upload=15830]: Next returned nil segment Aug 8 07:45:29.870622687 (15h18m43.457678938s) [upload=15830]: Waiting for error group managing segments... Aug 8 07:45:29.880490816 (15h18m43.467547067s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882136728 (15h18m43.469192969s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882913007 (15h18m43.469969248s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883383075 (15h18m43.470439315s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883463644 (15h18m43.470519886s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884241697 (15h18m43.471297938s) [noise=true , address=104.36.82.61:4375 , piece_id=VH56RRFT6UY2NHKAT2X63KKPPJSY2PF4KFYGFHQSFKIKMP7FY4TA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884323910 (15h18m43.471380151s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884973903 (15h18m43.472030154s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884984272 (15h18m43.472040514s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885552193 (15h18m43.472608444s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885640007 (15h18m43.472696247s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886142264 (15h18m43.473198504s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886250977 (15h18m43.473307217s) [noise=true , address=104.36.82.13:4215 , piece_id=MGZGACY7FQZKFOLCMDD2ABTLYKIVM7DM5XCPBTKRDRQOMLLYWXQA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887026404 (15h18m43.474082645s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887050258 (15h18m43.4741065s) [seg_pos={0 0}, upload=15830]: Segment reached optimal threshold of 14 pieces. Aug 8 07:45:29.887256934 (15h18m43.474313185s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N): protocol: expected piece hash; context canceled; EOF Aug 8 07:45:29.887313620 (15h18m43.474369862s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r5KA1p2BjwSqZvkS5skmU9t8WbF6rFkEqvvACPsfaPFuY6s3p): protocol: read tcp 5.161.89.252:46414->64.49.13.154:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887426782 (15h18m43.474483022s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1soiTQsiMNe5BkNHwwAGykmtP9LtLxncqE7Tv728ak8pjFeshg): protocol: read tcp 5.161.89.252:22516->64.49.13.26:4947: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887480762 (15h18m43.474537003s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: read tcp 5.161.89.252:16684->64.49.13.170:4717: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 8 07:45:29.887515146 (15h18m43.474571387s) [seg_pos={0 0}, upload=15830]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:45:29.887578214 (15h18m43.474634455s) [upload=15830]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:45:29.887584155 (15h18m43.474640396s) [upload=15830]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:30.055211210 (15h18m43.642267511s) [upload=15830]: Done uploading segments... Aug 8 07:45:30.055232540 (15h18m43.642288781s) [upload=15830]: Upload finished. err: <nil> Aug 8 07:46:24.069800557 (15h19m37.656856798s) [upload=15846]: Starting upload Aug 8 07:46:24.069887619 (15h19m37.65694386s) [upload=15846]: Done starting upload Aug 8 07:46:24.070014306 (15h19m37.657070546s) [upload=15846]: Uploading segments... Aug 8 07:46:24.070283849 (15h19m37.65734009s) [upload=15846]: Got next segment. Inline: false Aug 8 07:46:24.070335675 (15h19m37.657391917s) [upload=15846]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:24.070341606 (15h19m37.657397848s) [upload=15846]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:46:24.073193329 (15h19m37.66024957s) [upload=15846]: Next segment err: metaclient: Too Many Requests Aug 8 07:46:24.073210160 (15h19m37.660266402s) [upload=15846]: Done uploading segments... Aug 8 07:46:24.073226000 (15h19m37.660282241s) [upload=15846]: Upload finished. err: metaclient: Too Many Requests Duration: 624.127241ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 17.128831ms FAIL (2025-08-08 07:46:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== 30]: Uploading piece... Aug 8 07:45:29.750686944 (15h18m43.337743185s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.750696903 (15h18m43.337753144s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751077063 (15h18m43.338133314s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751096508 (15h18m43.338152759s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751438256 (15h18m43.338494498s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751454708 (15h18m43.338510948s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751803749 (15h18m43.33886s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.751812055 (15h18m43.338868295s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752268797 (15h18m43.339325038s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.752632205 (15h18m43.339688446s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753002305 (15h18m43.340058547s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753418863 (15h18m43.340475114s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.753859585 (15h18m43.340915837s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754388493 (15h18m43.341444735s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.754765928 (15h18m43.341822178s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.755148412 (15h18m43.342204653s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Uploading piece... Aug 8 07:45:29.870594195 (15h18m43.457650436s) [upload=15830]: Next returned nil segment Aug 8 07:45:29.870622687 (15h18m43.457678938s) [upload=15830]: Waiting for error group managing segments... Aug 8 07:45:29.880490816 (15h18m43.467547067s) [noise=true , address=45.91.116.10:4112 , piece_id=I7RDO6FUQM2ZYEE6IMPTQTLJZWJKDGZO4HVT3ISMZ6HYJRQNBZJQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882136728 (15h18m43.469192969s) [noise=true , address=104.36.82.37:4659 , piece_id=JV2A6Q6QEXZTTZVL2L2C7BDMFE22CAZMX7VKMLT36R5LAISK4GBA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.882913007 (15h18m43.469969248s) [noise=true , address=104.36.82.46:4491 , piece_id=YNFWBBV6ENYCDS5WTZOWSXBNUVFZLJVOTSRUQ4HVEWIINDMHI2OA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883383075 (15h18m43.470439315s) [noise=true , address=104.36.82.55:4209 , piece_id=OTICA7D2UWFFEHZ2K6Y4Q2B5VRNPE42P26XNDSGOKOU5VAMSRNUA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.883463644 (15h18m43.470519886s) [noise=true , address=104.36.82.43:4833 , piece_id=V6PLN3MBUU3GPVTFWKZ4LPSLLPXHJN2ISRERT2UZY5ZKGTM6M4BA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884241697 (15h18m43.471297938s) [noise=true , address=104.36.82.61:4375 , piece_id=VH56RRFT6UY2NHKAT2X63KKPPJSY2PF4KFYGFHQSFKIKMP7FY4TA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884323910 (15h18m43.471380151s) [noise=true , address=104.36.82.35:4598 , piece_id=6HHVL6NGE3LCRYRV2SLEENBHZ6XV7LLUZAKB5J6XERQU7MS3SYSA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884973903 (15h18m43.472030154s) [noise=true , address=104.36.82.58:4297 , piece_id=WRMYA5YY5LKJ5FUKS7NRCY326UDSZ7OGH7KKWHBVFQ3RQS5JH3EQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.884984272 (15h18m43.472040514s) [noise=true , address=104.36.82.60:4344 , piece_id=WLJLGJQRU2CGQEQSODRMVHBQ57QTTQITYSGPFRZD6XT3Z7N3JKRA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885552193 (15h18m43.472608444s) [noise=true , address=104.36.82.22:4541 , piece_id=SBM7YJNB6H6KB2UDPYY2ROYOM52SHBDVYGNSOCXFZ6Y3ARP3QVEA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.885640007 (15h18m43.472696247s) [noise=true , address=104.36.82.34:4569 , piece_id=VMTP6M25POHADXKX26GY35HR7ADXHDVBLEJ5VLOVBKE2TO3HDT2A, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886142264 (15h18m43.473198504s) [noise=true , address=104.36.82.36:4621 , piece_id=FWCH6W5TIT2XTEYXO6PKCV2J46ENXF5GOZKD2ZVUKETOBRIMPZFA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.886250977 (15h18m43.473307217s) [noise=true , address=104.36.82.13:4215 , piece_id=MGZGACY7FQZKFOLCMDD2ABTLYKIVM7DM5XCPBTKRDRQOMLLYWXQA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887026404 (15h18m43.474082645s) [noise=true , address=104.36.82.62:4443 , piece_id=C6BXLYUUKXD3JDPUP3XBXFYJDVMOYRJPKHC4NHL52FO4A3ZYO2PA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: <nil> Aug 8 07:45:29.887050258 (15h18m43.4741065s) [seg_pos={0 0}, upload=15830]: Segment reached optimal threshold of 14 pieces. Aug 8 07:45:29.887256934 (15h18m43.474313185s) [noise=true , address=104.36.82.3:4783 , piece_id=GR44SMMC2WRMQVXGQSPZVDCBAXOVQCEAVHMBDNYA6JBQQEBHPXWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ito72D7vikRdjK7BPt4LnhAK84bzi8AbZZPNWrTqmbjwQT1N): protocol: expected piece hash; context canceled; EOF Aug 8 07:45:29.887313620 (15h18m43.474369862s) [noise=true , address=64.49.13.154:4925 , piece_id=BSSS7UXJRW2LNZYEHVLRASILRAAFSITCIXJNLPE6CPAXYF3PMPMQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12r5KA1p2BjwSqZvkS5skmU9t8WbF6rFkEqvvACPsfaPFuY6s3p): protocol: read tcp 5.161.89.252:46414->64.49.13.154:4925: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887426782 (15h18m43.474483022s) [noise=true , address=64.49.13.26:4947 , piece_id=H45ZD23KK7V7TAZUC7SEC25PODF54XSSFTVWN3UCQPVX6HXPPZAA, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1soiTQsiMNe5BkNHwwAGykmtP9LtLxncqE7Tv728ak8pjFeshg): protocol: read tcp 5.161.89.252:22516->64.49.13.26:4947: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 8 07:45:29.887480762 (15h18m43.474537003s) [noise=true , address=64.49.13.170:4717 , piece_id=37M6NMF6THWN452TE3K7LA6BEXGSAY3AOBOSHJBY5DVWY65RHMWQ, seg_pos={0 0}, upload=15830]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a96r9qNJPvk95jgSAibJEaCDDisk6AFGHEC1XeXfhmQewg4ak): protocol: read tcp 5.161.89.252:16684->64.49.13.170:4717: use of closed network connection; conn closed: op; piecestore: piecestore close: context canceled Aug 8 07:45:29.887515146 (15h18m43.474571387s) [seg_pos={0 0}, upload=15830]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 07:45:29.887578214 (15h18m43.474634455s) [upload=15830]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 07:45:29.887584155 (15h18m43.474640396s) [upload=15830]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:45:30.055211210 (15h18m43.642267511s) [upload=15830]: Done uploading segments... Aug 8 07:45:30.055232540 (15h18m43.642288781s) [upload=15830]: Upload finished. err: <nil> Aug 8 07:46:24.069800557 (15h19m37.656856798s) [upload=15846]: Starting upload Aug 8 07:46:24.069887619 (15h19m37.65694386s) [upload=15846]: Done starting upload Aug 8 07:46:24.070014306 (15h19m37.657070546s) [upload=15846]: Uploading segments... Aug 8 07:46:24.070283849 (15h19m37.65734009s) [upload=15846]: Got next segment. Inline: false Aug 8 07:46:24.070335675 (15h19m37.657391917s) [upload=15846]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:46:24.070341606 (15h19m37.657397848s) [upload=15846]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 07:46:24.073193329 (15h19m37.66024957s) [upload=15846]: Next segment err: metaclient: Too Many Requests Aug 8 07:46:24.073210160 (15h19m37.660266402s) [upload=15846]: Done uploading segments... Aug 8 07:46:24.073226000 (15h19m37.660282241s) [upload=15846]: Upload finished. err: metaclient: Too Many Requests Duration: 17.150131ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 128.004921ms FAIL (2025-08-08 07:46:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 128.049905ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 163.760258ms FAIL (2025-08-08 07:45:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 163.785654ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 284.329872ms FAIL (2025-08-08 07:44:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 284.35529ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 136.36748ms FAIL (2025-08-08 07:44:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 136.38904ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 129.431034ms FAIL (2025-08-08 07:43:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 129.502018ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 421.801103ms FAIL (2025-08-08 07:43:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 421.826379ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 138.81028ms FAIL (2025-08-08 07:42:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 138.836599ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 144.567366ms FAIL (2025-08-08 07:41:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests Duration: 144.592061ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)