Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................X..............X.......................] duration: 156.775045ms FAIL (2025-08-17 13:17:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 156.800402ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XX.XX..............] duration: 571.183992ms FAIL (2025-08-17 13:17:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 571.21584ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................................XXXXXX.............] duration: 407.658566ms FAIL (2025-08-17 13:16:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 407.680888ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................................X...XXXX................] duration: 517.851156ms FAIL (2025-08-17 13:16:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 517.86855ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................X..............X......................] duration: 202.120152ms FAIL (2025-08-17 13:16:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 202.142444ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XX.XX.............] duration: 382.09584ms FAIL (2025-08-17 13:15:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 382.120286ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................................XXXXXX............] duration: 406.397339ms FAIL (2025-08-17 13:15:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 406.454625ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................................X...XXXX...............] duration: 589.822583ms FAIL (2025-08-17 13:15:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 589.85296ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X..............X.....................] duration: 135.419955ms FAIL (2025-08-17 13:15:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 135.438781ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XX.XX............] duration: 548.852357ms FAIL (2025-08-17 13:15:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 548.880291ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X..............X....................] duration: 533.248589ms FAIL (2025-08-17 13:15:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 533.272653ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X..............X...................] duration: 137.501681ms FAIL (2025-08-17 13:14:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 137.515307ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................................XXXXXX...........] duration: 416.290183ms FAIL (2025-08-17 13:14:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 416.322964ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................X...XXXX..............] duration: 500.660792ms FAIL (2025-08-17 13:14:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 500.684315ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................XX.XX...........] duration: 486.029899ms FAIL (2025-08-17 13:14:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 486.060398ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X..............X..................] duration: 144.798468ms FAIL (2025-08-17 13:14:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 144.825567ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................................XXXXXX..........] duration: 460.27934ms FAIL (2025-08-17 13:14:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 460.325807ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................X...XXXX.............] duration: 461.024223ms FAIL (2025-08-17 13:13:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 461.04438ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XX.XX..........] duration: 488.523611ms FAIL (2025-08-17 13:13:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 488.545181ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X..............X.................] duration: 132.611412ms FAIL (2025-08-17 13:13:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 132.633713ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XX.XX.........] duration: 881.58645ms FAIL (2025-08-17 13:13:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 881.613541ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................................X...XXXX............] duration: 452.040774ms FAIL (2025-08-17 13:13:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 452.066071ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................................XXXXXX.........] duration: 407.120192ms FAIL (2025-08-17 13:13:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 407.162252ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................XXXXXX........] duration: 789.647367ms FAIL (2025-08-17 13:12:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 789.677703ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X..............X................] duration: 428.922244ms FAIL (2025-08-17 13:12:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 428.981275ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................................X...XXXX...........] duration: 510.404751ms FAIL (2025-08-17 13:12:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 510.426221ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XX.XX........] duration: 654.800601ms FAIL (2025-08-17 13:12:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 654.826711ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X..............X...............] duration: 279.107552ms FAIL (2025-08-17 13:11:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 279.135325ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XXXXXX.......] duration: 380.332102ms FAIL (2025-08-17 13:11:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 380.355056ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X..............X..............] duration: 375.085634ms FAIL (2025-08-17 13:11:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 375.163241ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........................................X...XXXX..........] duration: 461.495097ms FAIL (2025-08-17 13:11:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 461.519041ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XX.XX.......] duration: 446.203875ms FAIL (2025-08-17 13:10:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 446.234763ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................XXXXXX......] duration: 399.549354ms FAIL (2025-08-17 13:10:44 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 399.574881ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................................X...XXXX.........] duration: 857.256546ms FAIL (2025-08-17 13:10:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 857.355842ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..............X.............] duration: 404.30504ms FAIL (2025-08-17 13:10:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 404.332702ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................................X...XXXX........] duration: 723.051654ms FAIL (2025-08-17 13:10:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 723.075379ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXXXXX.....] duration: 425.365074ms FAIL (2025-08-17 13:09:58 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 425.394569ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X..............X............] duration: 321.354973ms FAIL (2025-08-17 13:09:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 321.375351ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX.XX......] duration: 528.834417ms FAIL (2025-08-17 13:09:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 528.857169ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................................X...XXXX.......] duration: 625.16856ms FAIL (2025-08-17 13:09:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 625.194598ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................................XXXXXX....] duration: 513.195599ms FAIL (2025-08-17 13:09:16 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 513.231197ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................................X...XXXX......] duration: 589.321763ms FAIL (2025-08-17 13:08:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 589.371156ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XX.XX.....] duration: 878.949609ms FAIL (2025-08-17 13:08:50 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 878.975509ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X..............X...........] duration: 475.394373ms FAIL (2025-08-17 13:08:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 475.415393ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................................X...XXXX.....] duration: 876.341014ms FAIL (2025-08-17 13:08:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 876.363056ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XX.XX....] duration: 1.082560717s FAIL (2025-08-17 13:08:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.082581967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................................XXXXXX...] duration: 486.246085ms FAIL (2025-08-17 13:08:13 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Aug 17 13:05:26.773910806 (211h39m7.505178455s) [upload=220199]: Starting upload Aug 17 13:05:26.774013439 (211h39m7.505281089s) [upload=220199]: Done starting upload Aug 17 13:05:26.774108687 (211h39m7.505376347s) [upload=220199]: Uploading segments... Aug 17 13:05:26.774448908 (211h39m7.505716557s) [upload=220199]: Got next segment. Inline: false Aug 17 13:05:26.774529659 (211h39m7.505797309s) [upload=220199]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:26.774542563 (211h39m7.505810413s) [upload=220199]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:26.800669942 (211h39m7.531937602s) [seg_pos={0 0}, upload=220199]: Begin upload segment... Aug 17 13:05:26.800736297 (211h39m7.532003946s) [seg_pos={0 0}, upload=220199]: Done begin upload segment. Aug 17 13:05:26.800841635 (211h39m7.532109516s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800868956 (211h39m7.532136666s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.800950009 (211h39m7.532217658s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801393953 (211h39m7.532661603s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801475126 (211h39m7.532742776s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.801832077 (211h39m7.533099726s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802104890 (211h39m7.53337254s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802321588 (211h39m7.533589238s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802678157 (211h39m7.533945808s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.802690742 (211h39m7.533958392s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803042964 (211h39m7.534310614s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803191042 (211h39m7.534458693s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.803742099 (211h39m7.535009759s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804353918 (211h39m7.535621578s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.804900065 (211h39m7.536167716s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.805497930 (211h39m7.53676559s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Uploading piece... Aug 17 13:05:26.903223044 (211h39m7.634490693s) [upload=220199]: Next returned nil segment Aug 17 13:05:26.903279810 (211h39m7.63454746s) [upload=220199]: Waiting for error group managing segments... Aug 17 13:05:26.911780204 (211h39m7.643047853s) [noise=true , address=45.91.116.9:4262 , piece_id=ZTTEWA5X5O5WO5H7IFXOTYDIY7CVBEQMTAPTHNDJO2UZE7QVFM3Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912017279 (211h39m7.643284919s) [noise=true , address=45.91.116.6:4974 , piece_id=HLURL4RNIP2HSHV5TETIKIPLJKNOZKBR2SLRIKZCGY2MWWI7HABA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912520456 (211h39m7.643788106s) [noise=true , address=45.91.116.4:4940 , piece_id=QLTDLLP275BJQAN2CFMVNIK4K5NYR2ZGUV36HIERRCCFGAN6VH6Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.912758152 (211h39m7.644025793s) [noise=true , address=45.91.116.14:4191 , piece_id=VEMVIKQ666V2QWJQITHQHSBUQ6QLGLBIOJ2XQF2IXBO7BMMVTBWQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.913062364 (211h39m7.644330014s) [noise=true , address=45.91.116.9:4261 , piece_id=KBGNDO3RZHBHAWY2ETPNTZ5HDSGILE5X2NFYGLOMWMYPOVXWVUDA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.927349949 (211h39m7.6586176s) [noise=true , address=77.90.31.11:4555 , piece_id=FCHMEA7RHDWWOO3NAUKXKVVNT37MP6RQGZW4BD55IPNRLMEAK4AQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.929707954 (211h39m7.660975594s) [noise=true , address=45.91.116.14:4190 , piece_id=LC3C2G2UESMRQRTZMDYQKM54QDET22XUTADEGP2YGITU5YYXPBYA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.934481991 (211h39m7.665749641s) [noise=true , address=77.90.31.16:4733 , piece_id=TINS7Y5RCDRQUWZYYTV3IBHIZ2PG6LNITWVSEC4SL54IQKSMC5QA, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:26.937408663 (211h39m7.668676314s) [noise=true , address=77.90.31.18:4805 , piece_id=BKSD3HD6OUJZKB6TJG67TNPVVGVRN2GVQDQA34PNEPJAARFUKDOQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: <nil> Aug 17 13:05:56.761492801 (211h39m37.492760422s) [noise=true , address=104.36.82.8:4970 , piece_id=2Y34EPJ4LLNB4VUSI7TX6SPRDP2F256PUAHP5TACFXMFJJD57S5A, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761561431 (211h39m37.492829051s) [noise=true , address=104.36.82.15:4257 , piece_id=PQ7DGH24VD6XRZ63IFUMTLFII23H5XZF7NBIR4FDIJXMRGLTIC5Q, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: rpc: dial tcp 104.36.82.15:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761599212 (211h39m37.492866831s) [noise=true , address=104.36.82.9:4005 , piece_id=RKXWGUGRFMSMGUEZQOMOQTYVTUJTA7WARJRRUEJJSOIXMVON7KEQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: rpc: dial tcp 104.36.82.9:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761667450 (211h39m37.49293507s) [noise=true , address=104.36.82.12:4149 , piece_id=PQWQO44XQSYBHQG74FAOIPN2LKGYQ4BJK6UX3DI3YGE7T242ADKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: rpc: dial tcp 104.36.82.12:4149: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761701605 (211h39m37.492969224s) [noise=true , address=104.36.82.7:4910 , piece_id=2OWRVMHDDCWIAAETXMFAOSIVF5FXXTFFJ24WGDBFE6BDHGGEWIKQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761733755 (211h39m37.493001385s) [noise=true , address=104.36.82.5:4838 , piece_id=G3ZEAN7JTYRAI7LV5KDUE6W3CNLQHC4FNPODY76YKIWH52PJ7GBQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: rpc: dial tcp 104.36.82.5:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:05:56.761762949 (211h39m37.493030579s) [noise=true , address=104.36.82.2:4732 , piece_id=FTNHKCT5PLYXBMXLS4CA4ZGQYJ2A7QLH3TJI36NYVU5YXBGGEXDQ, seg_pos={0 0}, upload=220199]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:104.36.82.2:4732): protocol: rpc: dial tcp 104.36.82.2:4732: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Aug 17 13:05:56.761788137 (211h39m37.493055757s) [seg_pos={0 0}, upload=220199]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Aug 17 13:05:56.776656205 (211h39m37.507923835s) [upload=220199]: Done uploading segments... Aug 17 13:05:56.776672235 (211h39m37.507939865s) [upload=220199]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 486.273167ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XX.XX...] duration: 1.266235915s FAIL (2025-08-17 13:08:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 11h39m18.963301239s) [upload=220201]: Got next segment. Inline: false Aug 17 13:05:38.232064688 (211h39m18.963332307s) [upload=220201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:38.232068765 (211h39m18.963336385s) [upload=220201]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:38.260394616 (211h39m18.991662237s) [seg_pos={0 0}, upload=220201]: Begin upload segment... Aug 17 13:05:38.260422680 (211h39m18.99169031s) [seg_pos={0 0}, upload=220201]: Done begin upload segment. Aug 17 13:05:38.260441515 (211h39m18.991709135s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260457946 (211h39m18.991725576s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260717223 (211h39m18.991984853s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260731841 (211h39m18.991999471s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260959870 (211h39m18.9922275s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.260968356 (211h39m18.992235986s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261198057 (211h39m18.992465677s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261233694 (211h39m18.992501334s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261415967 (211h39m18.992683597s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261464098 (211h39m18.992731728s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261631563 (211h39m18.992899193s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261670385 (211h39m18.992938005s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261838482 (211h39m18.993106102s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.261906510 (211h39m18.99317414s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262041383 (211h39m18.993309003s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262119430 (211h39m18.993387071s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262267067 (211h39m18.993534687s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.262472443 (211h39m18.993740064s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Uploading piece... Aug 17 13:05:38.335674416 (211h39m19.066942037s) [upload=220201]: Next returned nil segment Aug 17 13:05:38.335685137 (211h39m19.066952757s) [upload=220201]: Waiting for error group managing segments... Aug 17 13:05:38.343418127 (211h39m19.074685747s) [noise=true , address=79.127.219.48:4101 , piece_id=FE5LSQUF7N42F5OZMKOVGT6YJY52SORGTVK6L432JIMIHXNSC6VQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.343638923 (211h39m19.074906542s) [noise=true , address=45.91.116.14:4192 , piece_id=3WIX7ZA43BIBVVZECKYAYSTRIH3VMPCK2OPB27TRFHZYPBUTJHKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.375313802 (211h39m19.106581432s) [noise=true , address=45.91.116.13:4173 , piece_id=NGIQVVZMXTHDYHUQUK3CKN5MMEKZBXQGAKYXZ6Q3AFSW7RBHQS6A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.384478504 (211h39m19.115746134s) [noise=true , address=77.90.31.12:4597 , piece_id=KEM4R5A2NNSX454JM4Z6YJZD2U7DXNCVFMSTSPULFAAWXA5TWSAQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.394391995 (211h39m19.125659625s) [noise=true , address=77.90.31.21:4882 , piece_id=HWQGRTQH45S4PLH37OZCMPRTQAOTWC6XWDU2MQKGTSMYZE3JU2TQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.395010046 (211h39m19.126277676s) [noise=true , address=45.91.116.10:4103 , piece_id=NOIXWLDGZHHCBFH74MNE2QYY7HUWYKE6FM7RADI2WMKW3NJZ7XYQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.402387698 (211h39m19.133655328s) [noise=true , address=45.91.116.11:4132 , piece_id=WG3XB5RFLVKWWTJOHMZIOLRCZI76TP6B5RSCXMIYAJ6YJ4CEIC6Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.409234503 (211h39m19.140502123s) [noise=true , address=45.91.116.7:4042 , piece_id=46YAQQV724C3JIUJNX7MI4WUDNVJZJYMG2RKQZN7TVLWKPBOJ22Q, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.428228938 (211h39m19.159496568s) [noise=true , address=79.127.220.97:4385 , piece_id=3NKAFBP245MT6PDEJ6EWBVZNEYJZ7MYAIGWFDPFEFXF47EZ4EW7A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.546929112 (211h39m19.278196732s) [noise=true , address=79.127.201.215:4356 , piece_id=LP3WWFCR4U54H5BUGUMOHAYZCAP7GKVWCCM7LMSIQO3XZEU6BHMQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.562922516 (211h39m19.294190176s) [noise=true , address=77.90.31.13:4625 , piece_id=JNAHBL4CT76MH6Z2MBL6DR4GFWHE6KXSYCW7PGA3R2JGCA4R4A3A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.786308181 (211h39m19.517575811s) [noise=true , address=79.127.205.233:4648 , piece_id=ISW4FQMK6FB45FXYFZGSAYUHDM7KRXRC2JEXIGC2ETC64NAWO6BA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:05:38.858771665 (211h39m19.590039295s) [noise=true , address=66.23.193.175:4563 , piece_id=ULXXDJ2UQHHUPBKSJBRL3VV2M6BTDMICQZG3NENPCX5235BMNVKQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: <nil> Aug 17 13:06:38.221240067 (211h40m18.952507717s) [noise=true , address=104.36.82.11:4122 , piece_id=WSGK65J4MUUMLDG2RBNSG3MMRI2IXEHVNPY7JGO6AJGTA4I6YARA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1oCu75zR8YccTunHjctcLzTrr9wTQqtAPXHiWrsQs41o8gxgh6, address:104.36.82.11:4122): protocol: rpc: dial tcp 104.36.82.11:4122: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221311331 (211h40m18.952578981s) [noise=true , address=104.36.82.58:4285 , piece_id=OQJLZYBUZ7SSYOYGPRUI6TAKAVDFZBFL7MKXIZWQPPHDSFNZFYIA, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:1rZhbymrDS6ahXLEqYRM64gcqRvG8CEnc8iGCTEhgXrpvRaQrp, address:104.36.82.58:4285): protocol: write tcp 5.161.89.252:35328->104.36.82.58:4285: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221354321 (211h40m18.952621971s) [noise=true , address=104.36.82.2:4759 , piece_id=WKBOMTCR5U46DYEUJL65GNZVT3TXZP7QFEL6TJ6PYBGLAIRRQWUQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12RFJ2jqXCSsXgc8ZDNTptWcaUV5Lm25Eobt58t7vGiQY34uhXz, address:104.36.82.2:4759): protocol: rpc: dial tcp 104.36.82.2:4759: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221450012 (211h40m18.952717662s) [noise=true , address=104.36.82.60:4343 , piece_id=NUKCLM5LQEACFEXGHTGLSAJTFAXQ7PN6EYB4UUUDJC64RKNKSS4A, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:16EsEMRW53sjKVag2WpSDGLhELyE2bSXyW4aAurCvDFKRNkXuf, address:104.36.82.60:4343): protocol: rpc: dial tcp 104.36.82.60:4343: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221534150 (211h40m18.952801799s) [noise=true , address=104.36.82.18:4376 , piece_id=6ODCXHQ7GIV3IHNCXEPEWBORVJJG644BCWTQE3LQDIQF3LT2JLDQ, seg_pos={0 0}, upload=220201]: Done uploading piece. err: ecclient: upload failed (node:12QfYPc1vbgwcnEWk67Bfbt5KiQ8sq322VKe951PFMTF6VaTzAR, address:104.36.82.18:4376): protocol: write tcp 5.161.89.252:49358->104.36.82.18:4376: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:38.221585135 (211h40m18.952852785s) [seg_pos={0 0}, upload=220201]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 17 13:06:38.236359888 (211h40m18.967627539s) [upload=220201]: Done uploading segments... Aug 17 13:06:38.236387120 (211h40m18.967654769s) [upload=220201]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.266260622s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X..............X..........] duration: 166.902084ms FAIL (2025-08-17 13:07:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 17 13:00:24.431681640 (211h34m5.162949261s) [upload=220113]: Starting upload Aug 17 13:00:24.431716026 (211h34m5.162983646s) [upload=220113]: Done starting upload Aug 17 13:00:24.431759107 (211h34m5.163026727s) [upload=220113]: Uploading segments... Aug 17 13:00:24.431790736 (211h34m5.163058366s) [upload=220113]: Got next segment. Inline: true Aug 17 13:00:24.431795495 (211h34m5.163063115s) [upload=220113]: Waiting for error group managing segments... Aug 17 13:00:24.431947481 (211h34m5.163215111s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:00:24.431953533 (211h34m5.163221163s) [upload=220113]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 13:00:24.431956157 (211h34m5.163223787s) [upload=220113]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 13:00:24.434700137 (211h34m5.165967787s) [upload=220113]: Done uploading segments... Aug 17 13:00:24.434709414 (211h34m5.165977034s) [upload=220113]: Upload finished. err: metaclient: Too Many Requests Duration:* 166.928385ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................................X...XXXX....] duration: 591.672911ms FAIL (2025-08-17 13:07:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== Aug 17 13:05:03.230244168 (211h38m43.961511788s) [upload=220194]: Starting upload Aug 17 13:05:03.230282931 (211h38m43.961550561s) [upload=220194]: Done starting upload Aug 17 13:05:03.230337775 (211h38m43.961605395s) [upload=220194]: Uploading segments... Aug 17 13:05:03.230498627 (211h38m43.961766257s) [upload=220194]: Got next segment. Inline: false Aug 17 13:05:03.230523233 (211h38m43.961790852s) [upload=220194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 13:05:03.230526769 (211h38m43.96179439s) [upload=220194]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 13:05:03.253244145 (211h38m43.984511775s) [seg_pos={0 0}, upload=220194]: Begin upload segment... Aug 17 13:05:03.253284361 (211h38m43.984551981s) [seg_pos={0 0}, upload=220194]: Done begin upload segment. Aug 17 13:05:03.253314057 (211h38m43.984581687s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253326811 (211h38m43.984594431s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253379720 (211h38m43.98464735s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253678482 (211h38m43.984946102s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253743043 (211h38m43.985010673s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.253961614 (211h38m43.985229234s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254079065 (211h38m43.985346696s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254257481 (211h38m43.985525111s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254408585 (211h38m43.985676204s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254523081 (211h38m43.985790721s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254695415 (211h38m43.985963045s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254849924 (211h38m43.986117554s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.254965993 (211h38m43.986233613s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255073975 (211h38m43.986341606s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255232403 (211h38m43.986500033s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255467977 (211h38m43.986735607s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255719861 (211h38m43.986987481s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.255961866 (211h38m43.987229496s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Uploading piece... Aug 17 13:05:03.323589897 (211h38m44.054857527s) [upload=220194]: Next returned nil segment Aug 17 13:05:03.323600959 (211h38m44.054868579s) [upload=220194]: Waiting for error group managing segments... Aug 17 13:05:03.351486953 (211h38m44.082754583s) [noise=true , address=45.91.116.10:4108 , piece_id=UJP6ESGMCDM52MG7CPCAGM6HFURCNFMC3MP6PF6VLTS2YKDH3ZWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.356136245 (211h38m44.087403865s) [noise=true , address=77.90.31.18:4838 , piece_id=ABIHECHOE7TXF7ZNEP45CMCAHWHRA4O6L4WLWXI3PRJHHDRU2OSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.359798501 (211h38m44.091066121s) [noise=true , address=77.90.31.16:4758 , piece_id=ZYUOJ663TRGB67BXC7LUGQRQGGSU56LD4TQLYR6E2R7CVQNWY2SQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.368712994 (211h38m44.099980613s) [noise=true , address=77.90.31.12:4624 , piece_id=QEDSYM53SGPAMLX7LNF4SGGUOHXX6VGVXBAYYY7HVG3D76HMYA6Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.451425619 (211h38m44.18269327s) [noise=true , address=77.90.31.13:4640 , piece_id=XLNBM6CDBXTG5ZRZFNBHAXV4CNLHBS4MM4MI5ATRINW6NYON4XBQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.488338257 (211h38m44.219605887s) [noise=true , address=45.91.116.6:4993 , piece_id=HGNWD6LQU7RF5JUT2DI5MFBBGIFN2QVTXBPAEIHINYQH5RSJMQ4Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.533602960 (211h38m44.26487062s) [noise=true , address=79.127.201.215:4349 , piece_id=TLF6ZGUDNPGNHU65NSB3MCVHELD5SRALOX5A5EYEMWZEA7CUNRZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.706703573 (211h38m44.437971194s) [noise=true , address=45.91.116.9:4284 , piece_id=7XM6UXEX7XCE53VUSVE5FVHOSLHZ3O7HINNBYIIPREWHZFTJBXQA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.845738394 (211h38m44.577006013s) [noise=true , address=77.90.31.11:4565 , piece_id=X5UWETN3VWNI5QELVHRWFIBVXX4XSBJ6PZYS3J3XLSKFKHK46KZA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:05:03.854673865 (211h38m44.585941515s) [noise=true , address=66.23.193.175:4562 , piece_id=K3MUJ7MX4WNQ6BIECEHIZJYEGB27VNUMF4KAEIRTQLMZ72BTCHGA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: <nil> Aug 17 13:06:03.213670588 (211h39m43.944938208s) [noise=true , address=104.36.82.12:4169 , piece_id=OR54CFSBKIN5VDRRPZZODSF4CFX75EUIC2GO25MW2B4UDY2NXM2Q, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: rpc: dial tcp 104.36.82.12:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213738616 (211h39m43.945006266s) [noise=true , address=104.36.82.60:4363 , piece_id=NJYMMSW7XMUSJVDILBJX3WEINUSHN2O5CIW2QTIUNI6SUJ2PLFOA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7, address:104.36.82.60:4363): protocol: write tcp 5.161.89.252:44086->104.36.82.60:4363: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213807617 (211h39m43.945075236s) [noise=true , address=104.36.82.44:4871 , piece_id=LRT2Z4WZ45CLC2QCR4WDEGQBER7SPXUPID3IAPQEQRBLIG3I5ZEQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx, address:104.36.82.44:4871): protocol: rpc: dial tcp 104.36.82.44:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213844345 (211h39m43.945111965s) [noise=true , address=104.36.82.11:4138 , piece_id=NMWQB3XGW7KJNHYI427JFDLXWLSGTFOG7NSCESBXAXDLIGKIWGWA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1H28XSTvFGa3XhavfhythHWs95cqjazu2RuqpsjNCmdSFvRLoh, address:104.36.82.11:4138): protocol: rpc: dial tcp 104.36.82.11:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213873840 (211h39m43.945141471s) [noise=true , address=104.36.82.9:4025 , piece_id=GX7UXJEE6TGXCE4VASFCTN7U4TU7UPFQ7XVEA5OFFRR7XRNKROIQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1244zx11aa7W2RGE7w1qeLfsHLxQ6f8tLJeNcxssa68QywFbX1R, address:104.36.82.9:4025): protocol: write tcp 5.161.89.252:16954->104.36.82.9:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213895070 (211h39m43.945162691s) [noise=true , address=104.36.82.17:4358 , piece_id=MTXF3E6B7UXC24I66GQF6ACWHVEYFHWBQOTGNZN3RXTVNFGITZIA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1FyLtRsiMJkXiZmQpvsVwUDUt3TunVJ5BpN61eeNqXddJbQGnK, address:104.36.82.17:4358): protocol: rpc: dial tcp 104.36.82.17:4358: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213912293 (211h39m43.945179912s) [noise=true , address=104.36.82.23:4462 , piece_id=NLJSKNDBAP73B4SPXIGBNYDVMACLZIQZ7ULDJE7YBAKKX53X5ZSQ, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu, address:104.36.82.23:4462): protocol: rpc: dial tcp 104.36.82.23:4462: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213916270 (211h39m43.94518389s) [noise=true , address=104.36.82.10:4101 , piece_id=3KVWATFDFOU6RQVO7UDOQGYKOFG4VK77LURI5DPEWIGE7F4RYHAA, seg_pos={0 0}, upload=220194]: Done uploading piece. err: ecclient: upload failed (node:1RmTDQrWoQBSpHrcUsb2PYijpcFdCyRya7VrMkxhWeQXdGeQpp, address:104.36.82.10:4101): protocol: rpc: dial tcp 104.36.82.10:4101: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 13:06:03.213938842 (211h39m43.945206472s) [seg_pos={0 0}, upload=220194]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 13:06:03.239464369 (211h39m43.970731999s) [upload=220194]: Done uploading segments... Aug 17 13:06:03.239484097 (211h39m43.970751716s) [upload=220194]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 591.695874ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)